CPSC 425: Computer Vision

Lecture 2: Image Formation

(unless otherwise stated slides taken or adapted from David Lowe, Bob Woodham, Jim Little, Fred Tung and Leon Sigal )

1
Menu for Today (January 12, 2022)
Topics:

— Image Formation — Projection

— Cameras and Lenses — Human eye (as camera)

Readings:

— Today’s Lecture: Forsyth & Ponce (2nd ed.) 1.1.1 — 1.1.3

— Next Lecture: Forsyth & Ponce (2nd ed.) 4.1, 4.5

Reminders:

— Complete Assignment 0 (ungraded) by Wed, January 19

— Google Colab Tutorials next week
— TA and Office hours are posted and2 will start on Monday, January 17
Today’s “fun” Example

Photo credit: reddit user Liammm

3
Today’s “fun” Example: Eye Sink Illusion

Photo credit: reddit user Liammm

4
Today’s “fun” Example: Eye Sink Illusion

“Tried taking a picture of a sink draining, wound up with a picture of an eye instead”
Photo credit: reddit user Liammm
5
Surprising faces!
Photo credit: camaro5.com
6
Lecture 1: Re-cap

Types of computer vision problems:

— Computing properties of the 3D world from visual data (measurement)
— Recognition of objects and scenes (perception and interpretation)
— Search and interact with visual data (search and organization)
— Manipulation or creation of image or video content (visual imagination)

Computer vision challenges:

— Fundamentally ill-posed
— Enormous computation and scale
— Lack of fundamental understanding of how human perception works

7
Lecture 1: Re-cap

Computer vision technologies have moved from research labs into

commercial products and services. Examples cited include:
— broadcast television sports
— electronic games (Microsoft Kinect)
— biometrics
— image search
— visual special effects
— medical imaging
— robotics
… many others

8
Related Disciplines
Artificial Intelligence (AI)
Robotics

Computer Vision
Machine Human
Learning Scope of CPSC 425 Computer
Interaction

Image Processing
Graphics Geometric Reasoning
Recognition Medical
Imaging
Computational
Photography Neuroscience

Optics

Slide Credit: James Hays (GA Tech) 9

Related Disciplines: Vision and Graphics

Images Model

Vision

Graphics

Inverse problems: analysis and synthesis

(it is sometimes useful to think about computer vision as inverse graphics)

Slide Credit: Kristen Grauman (UT Austin) 10

Why Study Computer Vision?

It is one of the most exciting areas of research in computer science

Among the fastest growing technologies in the industry today

11
12
Wired’s 100 Most Influential People in the World

13
14
CVPR Attendance

15
Lecture 2: Goal

To understand how images are formed

16
What is Computer Vision?
Compute vision, broadly speaking, is a research field aimed to enable computers to
process and interpret visual data, as sighted humans can.

Sensing Device Interpreting Device

Image (or video) Interpretation

Image Credit: https://www.flickr.com/photos/flamephoenix1991/8376271918 blue sky,

trees,
fountains,
UBC, …

17
Overview: Image Formation, Cameras and Lenses

The image formation process that produces a particular image depends on

— Lighting conditions
source sensor
— Scene geometry
— Surface properties
— Camera optics normal

— Sensor properties eye

surface
element

Sensor (or eye) captures the amount of light reflected from the object
18
(small) Graphics Review

source

normal sensor

surface
element

19
(small) Graphics Review
Surface reflection depends on both the viewing and illumination
direction, with Bidirectional Reflection Distribution Function:

source

normal sensor

Lambertian surface:
surface
element

constant, called albedo

20 Slide adopted from: Ioannis (Yannis) Gkioulekas (CMU)

(small) Graphics Review
Surface reflection depends on both the viewing and illumination
direction, with Bidirectional Reflection Distribution Function:

source

normal sensor

Lambertian surface:
surface
element

A Lambertian surface appears the same (brightness) from all directions.

21 Slide adopted from: Ioannis (Yannis) Gkioulekas (CMU)
 Lambertian sphere

“A surface that looks the same from any direction,

i.e., surface points are the same brightness”
(small) Graphics Review
Surface reflection depends on both the viewing and illumination
direction, with Bidirectional Reflection Distribution Function:

source

normal sensor

Lambertian surface:
surface
element

Mirror surface: all incident light reflected in one direction

23 Slide adopted from: Ioannis (Yannis) Gkioulekas (CMU)
Cameras

Old school film camera

Digital CCD/CMOS camera
Cameras

Old school film camera

Digital CCD/CMOS camera
Let’s say we have a sensor …

Digital CCD/CMOS camera

digital sensor
(CCD or
CMOS)

Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)

… and the object we would like to photograph
What would an image taken like this look like?

digital sensor
real-world
(CCD or
object
CMOS)

Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)

Bare-sensor imaging

digital sensor
real-world
(CCD or
object
CMOS)

Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)

Bare-sensor imaging

digital sensor
real-world
(CCD or
object
CMOS)

Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)

Bare-sensor imaging

digital sensor
real-world
(CCD or
object
CMOS)

Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)

Bare-sensor imaging

digital sensor
real-world
(CCD or
object
CMOS)

All scene points contribute to all sensor pixels

Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Bare-sensor imaging

All scene points contribute to all sensor pixels

Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Pinhole Camera

barrier (diaphragm)

pinhole
(aperture)
digital sensor
real-world
(CCD or
object
CMOS)

What would an image taken like this look like?

Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Pinhole Camera
most rays are
blocked

digital sensor
real-world
(CCD or
object
CMOS)

one makes it
through

Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)

Pinhole Camera

digital sensor
real-world
(CCD or
object
CMOS)

Each scene point contributes to only one sensor pixel

Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Camera Obscura (latin for “dark chamber”)

principles behind the

pinhole camera or
camera obscura were Mozi – 470 to 390 BCE
first mentioned by
Chinese philosopher
Mozi (Mo-Ti) (470 to
390 BCE)

Reinerus Gemma-Frisius observed an eclipse of the sun at Louvain on January

24, 1544. He used this illustration in his book, “De Radio Astronomica et
Geometrica,” 1545. It is thought to be the first published illustration of a camera
Credit: John H., Hammond, “Th Camera Obscure, A Chronicle”
obscura.
First Photograph on Record
La table servie

Credit: Nicéphore Niepce, 1822

39
Pinhole Camera

A pinhole camera is a box with a small hole (aperture) in it

Forsyth & Ponce (2nd ed.) Figure 1.2

Pinhole Camera

A pinhole camera is a box with a small hall (aperture) in it

Forsyth & Ponce (2nd ed.) Figure 1.2

Image Formation

Forsyth & Ponce (2nd ed.) Figure 1.1

Credit: US Navy, Basic Optics and Optical Instruments. Dover, 1969

42
Elsa Dorfman and the large format camera

Credit: TuftsNow
Accidental Pinhole Camera

The pin hole camera is not just an abstraction, these are images of
“accidental” room-sized pinhole cameras that resulted from having blinds
with small holes in them.

44 Image Credit: Ioannis (Yannis) Gkioulekas (CMU)

What’s this?
Pinhole Camera (Simplified)

f’ is the focal length of the camera

Pinhole Camera (Simplified)

f’ is the focal length of the camera

Note: In a pinhole camera we can adjust the focal length, all this will do is change the size of the resulting image
Pinhole Camera (Simplified)
It is convenient to think of the image plane which is in from of the pinhole

What happens if object moves towards the camera? Away from the camera?
48
Perspective Effects
Far objects appear smaller than close ones

Forsyth & Ponce (2nd ed.) Figure 1.3a

Perspective Effects
Far objects appear smaller than close ones

Forsyth & Ponce (2nd ed.) Figure 1.3a

Perspective Effects
Far objects appear smaller than close ones

Forsyth & Ponce (2nd ed.) Figure 1.3a

Size is inversely proportional to distance

Perspective Effects
Parallel lines meet at a point (vanishing point)

Forsyth & Ponce (1st ed.) Figure 1.3b

52
Vanishing Points
Each set of parallel lines meets at a different point
— the point is called the vanishing point

53
Vanishing Points
Each set of parallel lines meet at a different point
— the point is called vanishing point

Sets of parallel lines one the same plane lead to collinear vanishing points
— the line is called a horizon for that plane

54
Vanishing Points

Slide Credit: David Jacobs

55
Vanishing Points

Slide Credit: David Jacobs

56
Vanishing Points
Each set of parallel lines meet at a different point
— the point is called vanishing point

Sets of parallel lines one the same plane lead to collinear vanishing points
— the line is called a horizon for that plane

Good way to spot fake images

— scale and perspective do not work
— vanishing points behave badly

57
Vanishing Points

One point perspective

Two point perspective

Vanishing Vanishing
point point

58 Slide Credit: Efros (Berkeley), photo from Criminisi

Perspective Aside

Image credit: http://www.martinacecilia.com/place-vanishing-points/

60
Perspective Aside

Image credit: http://www.martinacecilia.com/place-vanishing-points/

61
Properties of Projection

— Points project to points

— Lines project to lines
— Planes project to the whole or half image
— Angles are not preserved
— Incidences (intersections) are preserved

62
Properties of Projection

— Points project to points

— Lines project to lines
— Planes project to the whole or half image
— Angles are not preserved

Degenerate cases
— Line through focal point (pinhole or aperture) projects to a point
— Plane through focal point projects to a line

63
Projection Illusion

64
Perspective Projection

3D object point
Forsyth & Ponce (1st ed.) Figure 1.4

projects to 2D image point where

Note: this assumes world coordinate frame at the optical center (pinhole) and aligned with the image plane,
image coordinate frame aligned with the camera coordinate frame
67
Perspective Projection: Proof

3D object point
Forsyth & Ponce (1st ed.) Figure 1.4

projects to 2D image point where

Note: this assumes world coordinate frame at the optical center (pinhole) and aligned with the image plane,
image coordinate frame aligned with the camera coordinate frame
68
Aside: Camera Matrix
Camera Matrix

3D object point
Forsyth & Ponce (1st ed.) Figure 1.4

projects to 2D image point where

Note: this assumes world coordinate frame at the optical center (pinhole) and aligned with the image plane,
image coordinate frame aligned with the camera coordinate frame
Aside: Camera Matrix
Camera Matrix

projects to 2D image point where

71
Aside: Camera Matrix
Camera Matrix

projects to 2D image point where

72
Aside: Camera Matrix
Camera Matrix

Pixels are squared / lens is perfectly symmetric

Sensor and pinhole perfectly aligned
Coordinate system centered at the pinhole

projects to 2D image point where

73
Aside: Camera Matrix
Camera Matrix

Pixels are squared / lens is perfectly symmetric

Sensor and pinhole perfectly aligned
Coordinate system centered at the pinhole

projects to 2D image point where

74
Aside: Camera Matrix
Camera Matrix

Pixels are squared / lens is perfectly symmetric

Sensor and pinhole perfectly aligned
Coordinate system centered at the pinhole

projects to 2D image point where

75
Aside: Camera Matrix
Camera Matrix

Pixels are squared / lens is perfectly symmetric

Sensor and pinhole perfectly aligned
Coordinate system centered at the pinhole

projects to 2D image point where

78
Aside: Camera Matrix
Camera Matrix

Camera calibration is the process of estimating parameters of the

camera matrix based on set of 3D-2D correspondences
(usually requires a pattern whose structure and size is known)

projects to 2D image point where

79
Perspective Projection

3D object point
Forsyth & Ponce (1st ed.) Figure 1.4

projects to 2D image point where

Note: this assumes world coordinate frame at the optical center (pinhole) and aligned with the image plane, image coord
Weak Perspective

Forsyth & Ponce (1st ed.) Figure 1.5

3D object point in projects to 2D image point

where and
81
Orthographic Projection

Forsyth & Ponce (1st ed.) Figure 1.6

3D object point projects to 2D image point

where
82
Summary of Projection Equations
3D object point projects to 2D image point where

Perspective

Weak Perspective

Orthographic
83
Projection Models: Pros and Cons

Weak perspective (including orthographic) has simpler mathematics

— accurate when object is small and/or distant
— useful for recognition

Perspective is more accurate for real scenes

When maximum accuracy is required, it is necessary to model additional

details of a particular camera
— use perspective projection with additional parameters (e.g., lens distortion)

84
Reminders

Readings:

— Today’s Lecture: Forsyth & Ponce (2nd ed.) 1.1.1 — 1.1.3

— Next Lecture: Forsyth & Ponce (2nd ed.) 4.1, 4.5
Szeliski (background): 2.1, 2.1.1, 2.1.5, 2.2.1, 2.2.2

Reminders:

— Complete Assignment 0 (ungraded) by Wed, Jan 19

— WWW: http://www.cs.ubc.ca/~little/cpsc425/assignments/Assignment0.html

85