CS-474 Lecture 1

Introduction
Ahmad Rauf Subhani, Ph.D.
[email protected]

1
 What is a
Digital Image ?
• An image is a 2D projection of the 3D scene
• Mathematically, an image is a 2D function
f(x,y), where x and y are spatial (plane)
coordinates, and f(x,y) is the amplitude of
the function at (x,y) called intensity or gray
level
• x, y and f(x,y) are all discrete quantities for
a digital image
• An image point at (x,y) is called a picture
element, image element, pel or pixel
2
 What is a Digital Image?

What we see What a computer sees

3
 What is Image Processing ?
Image processing is a method to perform some operations on an
image, in order to get an enhanced image or to extract some useful
information from it

4
 What is Digital Image Processing ?

• Digital image processing is the transformation of

a digital image by means of a digital computer
(this is what we are studying in this course)
• Image processing starts with an image and
(usually) produces a modified version of that
image – e.g., enhancement, restoration,
compression, i.e., the output is also an image
• Machine vision / computer vision aims to
understand the content of images at a higher
level so as to take decisions based on visual input

5
Image Acquisition

6
Sampling and Quantization

7
 Quantization and Sampling
Result
 Digital Image is an approximation of a real world
scene

8
8
 Digital Image Representation

 Number of bits required to store an image

b  M N k
 Image having 2k intensity levels
 k – bit depth of an image
 256 intensity levels => 8-bit image

9
9
 Coordinate Convention

Standard convention in programming is to place

the image origin at top, left corner 10
 What is Digital Image Processing ?

• Image processing is the transformation of f(x,y)

by some means (e.g., optical transformation)
• Digital image processing is the transformation of
a digital image by means of a digital computer
• Image processing starts with an image and
(usually) produces a modified version of that
image – e.g., enhancement, restoration,
compression, i.e., the output is also an image
• Machine vision / computer vision aims to
understand the content of images at a higher
level so as to take decisions based on visual input
11
 From Image Processing to Computer Vision:
 low, mid and high-level processes

Low Level Process

Input: Image
Output: Image
Examples: Noise
removal, image
sharpening, etc.

Image Processing
12
Example: Low Level Processing

13
 From Image Processing to Computer Vision:
 low, mid and high-level processes

Low Level Process Mid Level Process

Input: Image Input: Image
Output: Image Output: Attributes
Examples: Noise Examples: feature
removal, image extraction,
sharpening, etc. segmentation, motion,
appearance and
texture perception, etc.
Image Processing Early (computer) vision
14
Example: Mid Level Processing

Segmentation of image into regions

15
 From Image Processing to Computer Vision:
 low, mid and high-level processes

Low Level Process Mid Level Process High Level Process

Input: Image Input: Image Input: Attributes/Image
Output: Image Output: Attributes Output: Understanding
Examples: Noise Examples: feature Examples: Object
removal, image extraction, recognition, scene
sharpening, etc. segmentation, motion, understanding, shape
appearance and from motion, texture,
texture perception, etc. stereo, stitching, etc.
Image Processing Early (computer) vision Computer Vision, Pattern
Recognition, Machine Learning16
Example: High Level Processing

17
Example: High Level Processing

Robot Navigation
18
 What is (computer) vision?

Image (or video) Sensing device Interpreting device Interpretations

garden, spring,
bridge, water,
trees, flower,
green, etc.

19
The goal of computer vision
• To bridge the gap between pixels and “meaning”

Source: S. Narasimhan
What we see What a computer sees
20
Related Disciplines

Artificial
intelligence Machine
Graphics learning
Computer
Image vision Neuro-
processing science
Algorithms

21
Vision and graphics

Images Vision Model

Graphics

Converse problems: analysis and synthesis.

22
 Why vision?
• Images and video are everywhere!

Personal photo albums Movies, news, sports

Surveillance and security Medical and scientific images 23

Slide credit; L. Lazebnik
1981: Nobel Prize in medicine

24
25

Source: Fei-Fei Li & Ruohan Gao & Yunzhu Li

 First PhD thesis of computer vision

Block World

Source: Fei-Fei Li & Ruohan Gao & Yunzhu Li

26
27

Source: Fei-Fei Li & Ruohan Gao & Yunzhu Li

28

Source: Fei-Fei Li & Ruohan Gao & Yunzhu Li

29

Source: Fei-Fei Li & Ruohan Gao & Yunzhu Li

30
31
32
A transfer from the basic science research to the real-world application

33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
Vision for measurement
Multi-view stereo for
Real-time stereo Structure from motion community photo collections

NASA Mars Rover

Pollefeys et al.
Goesele et al.

52
Slide credit: L. Lazebnik
Current state of the art
• The next few slides show some examples of what
current vision systems can do…

53
Optical character recognition (OCR)
Technology to convert scanned docs to text

Digit recognition, AT&T labs License plate readers

http://www.research.att.com/~yann/ http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

54
 Face detection

• Many (if not all) new digital cameras now detect faces
• Camera can then set the focus and appropriate exposure for the shot
automatically
• Aka face-priority auto focus (AF)
• Further applications: take shot only when a certain number of faces
visible (instead of auto-timer) / when all faces smiling, etc. 55
Sony Cyber-shot® T70 Digital Still Camera 56
 The remarkable story of
Vision-based biometrics Sharbat Gula, first
photographed in 1984 aged
12 in a refugee camp in
Pakistan by National
Geographic photographer
Steve McCurry, and traced 18
years later to a remote part of
Afghanistan where she was
again photographed by
McCurry…

National Geographic accepted

“How the Afghan Girl was Identified by Her Iris
and published this conclusion
Patterns” Read the story
in a second cover issue
featuring Sharbat Gula, 18
years after the first, and the
Society launched their
"Afghan Girl's Fund" to assist
the education of Muslim girls
in cultures that discourage or
prohibit female education.
57
Login without a password…

Face recognition systems now

Fingerprint scanners on
beginning to appear more widely
many new laptops, http://www.sensiblevision.com/
other devices

58
Object recognition (in mobile phones)

• This is becoming real:

• Microsoft Research
• Point & Find, Nokia
• SnapTell.com (now amazon)
59
Special effects: shape capture

The Matrix movies, ESC Entertainment, XYZRGB, NRC

60
Special effects: motion capture
Sports
 Slide content courtesy of Amnon Shashua

Smart cars

• Mobileye [wiki article]

• Vision systems currently in many car models
Vision in space

NASA'S Mars Exploration Rover Spirit captured this westward view from atop
a low plateau where Spirit spent the closing months of 2007.

Vision systems (JPL) used for several tasks

• Panorama stitching
• 3D terrain modeling
• Obstacle detection, position tracking
• For more, read “Computer Vision on Mars” by Matthies et al.
Industrial robots

Vision-guided robots position nut runners on wheels

Medical imaging

Image guided surgery

3D imaging
Grimson et al., MIT
MRI, CT
 Vision-based HCI

Digimask: put your face on a 3D avatar.

Nintendo Wii has camera-based IR

tracking built in. See Lee’s work at
CMU on clever tricks on using it to
create a multi-touch display!

“Game turns moviegoers into Human Joysticks”, CNET

Camera tracking a crowd, based on this work. 67
 Robotics

NASA’s Mars Spirit Rover http://www.robocup.org/

http://en.wikipedia.org/wiki/Spirit_rover

68
Current state of the art
• You just saw examples of current systems.
• Many of these are less than 10 years old

• This is a very active research area, and rapidly changing

• Many new applications in the next 10 years

• To learn more about vision applications and companies

• David Lowe maintained an excellent overview of vision
companies (last updated 2015)
• http://www.cs.ubc.ca/spider/lowe/vision.html

69