Introduction to Computer ◼ Vision: an information source

Vision 1. A source of information about the

surrounding world
❑ Object identity
❑ Object location
❑ Object dynamics

Introduction to CV 2

Vision

2. Supports intelligent interaction with the ◼ What does it mean

environment ❑ to see?
❑ Navigation ❑ “to know what is where
❑ Manipulation by looking”.
❑ Decision making
3. Information derived without physical contact
❑ Optical data is acquired at a distance. ---Aristotle’s definition
❑ Enables unobtrusive sensing.

Introduction to CV 3 Introduction to CV 4
 Definition: What is computer vision?

◼ Vision allows humans to perceive and understand ◼ Computer vision aims to duplicate the effect of
the world surrounding us. human vision by electronically perceiving and
understanding an image.
◼ How to discover from images what is present in ❑ The study of recovering useful properties of the world
(what, where)
the world, where things are, what actions are
❑ from one or more images (by looking)
taking place.
❑ with an algorithmic level of specification

Introduction to CV 5 Introduction to CV 6

Computer Vision, Also Known As ...

◼ Deals with the development of the theoretical ◼ Image Analysis

and algorithmic basis by which useful ◼ Scene Analysis
information about the 3D world can be ◼ Image Understanding
automatically extracted and analyzed from a
single or multiple of 2D images of the world.

Introduction to CV 7 Introduction to CV 8
 Some Related Disciplines

◼ Problems the computer vision solves ◼ Image processing

❑ Computing properties of the world from one or more ◼ Pattern recognition
images
◼ Computer graphics
❑ Properties of interest:
◼ geometric (shape, position), ◼ Robotics
◼ photometric (surface reflectance) ◼ Artificial Intelligence
◼ dynamic (velocity)

Introduction to CV 9 Introduction to CV 10

Image processing

◼ Subject
❑ Generation of new
Image
images from existing processing
Image Enhancement
images.
❑ Images altered in
some desired fashion.
Input image Output image

Introduction to CV 11 Introduction to CV 12
 Image Restoration (e.g., correcting out-focus images) ◼ Image Compression

Introduction to CV 13 Introduction to CV 14

Pattern recognition

◼ Examples ◼ Subject
❑ noise suppression ❑ Classification of patterns
❑ feature enhancement ❑ Pattern represented by a set Pattern
recognition
❑ video stabilization of numbers representing
characteristics of an object
◼ Relationship to computer vision (e.g., height, weight)
❑ Often serves to provide components to computer vision. Feature vector Object class
❑ Preprocessing of sensed data.

Introduction to CV 15 Introduction to CV 16
◼ Has a very long history (research work in this field started ◼ Examples
in the 60s). ❑ Classification of chemical composition from spectral
◼ Concerned with the recognition and classification of 2D measurements.
objects mainly from 2D images. ❑ Classification of disease from symptoms.
◼ Many classic approaches only worked under very ❑ Classification of targets from visual features.
constrained views (not suitable for 3D objects). ◼ Relationship to computer vision
◼ It has triggered much of the research which led to today’s ❑ Techniques of pattern recognition can usefully be
field of computer vision. applied to the output of a computer vision system.
◼ Many pattern recognition principles are used extensively Capable of assigning imaged objects to classes based
on vision processing.
in computer vision.

Introduction to CV 17 Introduction to CV 18

Computer graphics

◼ Subject
❑ Generation of images
from models or other Computer
graphics
computational
specification.
❑ Models to Images Model Image

Computer vision

Introduction to CV 19 Introduction to CV 20
◼ Examples
❑ Photorealistic rendering (真實感渲染)
❑ Computer animation
❑ Abstract design
◼ Relationship to computer vision
❑ Akin to an inverse
❑ Potential to combine together, e.g., image based
rendering (IBR)

Introduction to CV 21 Introduction to CV 22

Robotic Vision Artificial Intelligence

◼ Application of computer vision in robotics. ◼ Concerned with designing systems that are intelligent and
with studying computational aspects of intelligence.
◼ Some important applications include :
◼ It is used to analyze scenes by computing a symbolic
❑ Autonomous robot navigation
representation of the scene contents after the images
❑ Inspection and assembly have been processed to obtain features.
◼ Many techniques from artificial intelligence play an
important role in many aspects of computer vision.
◼ Computer vision is considered a sub-field of artificial
intelligence.

Introduction to CV 23 Introduction to CV 24
Computer Vision

◼ Giving computers the ability to see is not an easy

task - we live in a three dimensional (3D) world,
and when computers try to analyze objects in 3D
space, available visual sensors (e.g., TV cameras)
usually give two dimensional (2D) images, and
this projection to a lower number of dimensions
incurs an enormous loss of information.
Images to Models

Introduction to CV 25 Introduction to CV 26

Why is Computer Vision Difficult?

◼ It is a many-to-one mapping ◼ It is computationally intensive

❑ A variety of surfaces with different material and ◼ We do not understand the recognition problem
geometrical properties, possibly under different
lighting conditions, could lead to identical images
❑ Inverse mapping has non unique solution (a lot of
information is lost in the transformation from the 3D
world to the 2D image)

Introduction to CV 27 Introduction to CV 28
Practical Considerations

◼ Impose constraints to recover the scene ◼ Industrial computer vision systems work very
❑ Gather more data (images) well
❑ Make assumptions about the world ❑ Make strong assumptions about lighting conditions
◼ Computability and robustness ❑ Make strong assumptions about the position of objects
❑ Is the solution computable using reasonable resources? ❑ Make strong assumptions about the type of objects
❑ Is the solution robust? (same conclusion under different
input condition)

Introduction to CV 29 Introduction to CV 30

An Industrial Computer Vision System

◼ In order to simplify the task of computer vision

understanding, three levels are usually
distinguished
❑ Low level image processing
❑ Intermediate-level image processing
❑ High level image understanding.

Introduction to CV 31 Introduction to CV 32
Low level digital image processing

◼ Low level computer vision techniques overlap ◼ Standard procedures are applied to improve image
almost completely with digital image processing, quality
which has been practiced for decades. ◼ Procedures are required to have no intelligent
◼ Low level methods usually use very little capabilities
knowledge about the content of images.

Introduction to CV 33 Introduction to CV 34

Intermediate-level processing

The following sequence of processing steps is ◼ Extract and characterize components in the image
commonly recognized: ◼ Some intelligent capabilities are required.
◼Image Acquisition:
❑ An image is captured by a sensor (such as a TV
camera) and digitized;
◼Preprocessing:
❑ Computer suppresses noise (image pre-processing)
and maybe enhances some object features which are
relevant to understand the image

Introduction to CV 35 Introduction to CV 36
 High-level image understanding

◼ Image segmentation: ◼ High level processing is based on knowledge,

❑ Edge detection goals, and plans of how to achieve those goals.
❑ Computer tries to separate objects from the image ◼ Artificial intelligence (AI) methods are used in
background. many cases.
◼ Object description and classification in a totally ◼ High level computer vision tries to imitate human
segmented image cognition and the ability to make decisions
according to the information contained in the
image.

Introduction to CV 37 Introduction to CV 38

Recognition Cues
◼ How are we able to discern reality and an image of reality?
◼ Scene interpretation, even of complex, cluttered scenes is a What clues are present in the image?
straightforward task for humans. What knowledge do we use to process this image?

Introduction to CV 39 Introduction to CV 40
The role of color The role of texture
◼ What is this object? ◼ Characteristic image texture can help us readily
Does color play a role in recognition?
recognize objects.
Might this be easier to recognize from a different view?

Introduction to CV 41 Introduction to CV 42

The role of shape The role of grouping

Introduction to CV 43 Introduction to CV 44
Mathematics in Computer Vision Computer Vision Applications
◼ In the early days of computer vision, vision systems employed ◼ Industrial inspection
simple heuristic methods.
◼ Today, the domain is heavily inclined towards theoretically, well- ◼ Surveillance, monitoring and security
founded methods involving non-trivial mathematics.
❑ Calculus
◼ Person recognition (automated fingerprint, face,
❑ Linear Algebra iris,…)
❑ Probabilities and Statistics
❑ Signal Processing
◼ Human computer interface (Gesture recognition)
❑ Projective Geometry
❑ Computational Geometry
❑ Optimization Theory
❑ Control Theory

Introduction to CV 45 Introduction to CV 46

Visual Inspection

◼ Autonomous vehicles
◼ Hand-eye robotics
◼ Medical image analysis
◼ Image databases
◼ Space applications
◼ Virtual reality

Introduction to CV 47 Introduction to CV 48
Industrial robots Aerial surveillance

◼ Real-world needs
❑ acquisition of information about
ground activities
❑ from well positioned mobile
platform, drone
◼ State of the art example
❑ video geolocation
❑ alignment of video to calibrated
reference imagery to attain
geodetic coordinates
Vision-guided robots position nut runners on wheels

Introduction to CV 49 Introduction to CV 50

Personal identification
◼ Real-world needs
❑ secure access control to
sensitive areas and
materials reliable personal
verification and
identification：biometric “How the Afghan Girl was Identified by Her Iris Patterns” Read the story
wikipedia
identification
◼ State of the art example
❑ biometric-based
identification
❑ automated iris recognition

Introduction to CV 51 Introduction to CV 52
 ◼ Fingerprint Verification / Identification

Introduction to CV 53 Introduction to CV 54

Login without a password…

◼ Fingerprint Identification Research

Face recognition systems now

Fingerprint scanners on
beginning to appear more widely
many new laptops, http://www.sensiblevision.com/
other devices

Introduction to CV 55 Introduction to CV 56
Face Detection

Introduction to CV 57 Introduction to CV 58

◼ Face Detection/Recognition Research

◼ Many new digital cameras and mobile phones

now detect faces
❑ Canon, Sony, Fuji, …

Introduction to CV 59 Introduction to CV 60
Facial Expression Recognition Smile detection

Sad、Happy、Fear、Disgust、Surprise、Angry

Introduction to CV 61 Introduction to CV 62

Human computer interface Vision-based interaction (and games)

◼ Real-world needs
❑ More natural interfaces
between humans and
computers (and other artifacts)
Increased speed and ease of
interaction
◼ State of the art example
❑ Automated hand gesture
recognition. Digimask: put your face on a 3D avatar.
❑ Vision-based detection,
localization and tracking

Introduction to CV 63 Introduction to CV 64
 Medical Medical Applications

◼ Real-world needs skin cancer breast cancer

❑ aides to physicians in
diagnosis of disease tools for
increasing accuracy and
throughput
◼ State of the art examples
❑ shape-based lesion （損傷）
detection
❑ alignment of real-time retinal
(視網膜) imagery with
previous reference
❑ change detection for cancer
detection

Introduction to CV 65 Introduction to CV 66

Character Recognition: vowel

Image guided surgery： Minimally invasive

Introduction to CV 67 Introduction to CV 68
Optical character recognition (OCR) Document Handling
◼ Technology to convert
scanned docs to text
• If you have a scanner, it
probably came with OCR
software

License plate readers

http://en.wikipedia.org/wiki/Automatic_number_p
late_recognition

Introduction to CV 69 Introduction to CV 70

Signature Verification Object Recognition

Introduction to CV 71 Introduction to CV 72
Object Recognition Research Object recognition (in supermarkets)
reference view 1 reference view 2
“A smart camera is flush-mounted in
the checkout lane, continuously
watching for items. When an item is
detected and recognized, the cashier
verifies the quantity of items that were
found under the basket, and continues
to close the transaction. The item can
remain under the basket, and with
LaneHawk, you are assured to get
paid for it… “

novel view recognized

Introduction to CV 73 Introduction to CV 74

Object recognition (in mobile Indexing into Databases

phones) ◼ Shape content

Introduction to CV 75 Introduction to CV 76
Indexing into Databases (cont’d) Target Recognition

◼ Color, texture ◼ Department of Defense (Army, Airforce, Navy)

Introduction to CV 77 Introduction to CV 78

Interpretation of Aerial Photography Autonomous Vehicles

◼ Interpretation of aerial photography is a problem domain in
both computer vision and photogrammetry （攝影學）. ◼ Land, Underwater, Space

Introduction to CV 79 Introduction to CV 80
Smart cars Traffic Monitoring

Introduction to CV 81 Introduction to CV 82

Human Activity Recognition Astronomy Applications Research

Introduction to CV 83 Introduction to CV 84
 Morphing (變形)

From person to gorilla

NASA'S Mars Exploration Rover Spirit captured this westward view from atop
a low plateau where Spirit spent the closing months of 2007.

Introduction to CV 85 Introduction to CV 86

Inserting Artificial Objects into a

Scene

Introduction to CV 87