Computer Vision - 01 Introduction
Computer Vision - 01 Introduction
(240069)
Introduction
Textbook
Computer Vision: Algorithms and Applications by R. Szeliski
(http://szeliski.org/Book/)
Additional References
Computer vision: models, learning and inference by Simon J.D. Prince
(http://www.computervisionmodels.com/)
Computer Vision: A Modern Approach by D. Forsyth and J. Ponce
Multiple View Geometry by R. Hartly and A. Zisserman
2
What is Computer Vision?
Vision
It is the process of discovering what is present in the world and where
it is by looking.
Computer Vision
It is the study of analysis of pictures and videos in order to achieve
results similar to those as by men.
3
What is Computer Vision?
Pattern Machine
Recognition Learning
Mathematics
Image Computer & Probability
Processing Vision
Computer Optics
Graphics
4
Contents
6
Image Processing
Point operators
Pixel transforms
Color transforms
Compositing and matting
Histogram equalization
Application: Tonal adjustment
Linear filtering
Separable filtering
Examples of linear filtering
Band-pass and steerable filters
More neighborhood operators
Non-linear filtering
Morphology
Distance transforms
Connected components
7
Feature Detection and Matching
Points and patches
Feature detectors
Feature descriptors
Feature matching
Feature tracking
Application: Performance-driven animation
Edges
Edge detection
Edge linking
Application: Edge editing and enhancement
Lines
Successive approximation
Hough transforms
Vanishing points
Application: Rectangle detection
8
Segmentation
Active contours
Snakes
Dynamic snakes and
CONDENSATION
Scissors
Level Sets
Application: Contour tracking and
rotoscoping
Split and merge
Watershed
Region splitting (divisive clustering)
Region merging (agglomerative
clustering)
Graph-based segmentation
Probabilistic aggregation
Mean shift and mode finding
K-means and mixtures of Gaussians
Mean shift
Normalized cuts
Graph cuts and energy-based methods
Application: Medical image
segmentation
9
Segmentation
Active contours
Snakes
Dynamic snakes and
CONDENSATION
Scissors
Level Sets
Application: Contour tracking and
rotoscoping
Split and merge
Watershed
Region splitting (divisive clustering)
Region merging (agglomerative
clustering)
Graph-based segmentation
Probabilistic aggregation
Mean shift and mode finding
K-means and mixtures of Gaussians
Mean shift
Normalized cuts
Graph cuts and energy-based methods
Application: Medical image
segmentation
10
Feature-based alignment
2D and 3D feature-based alignment
2D alignment using least squares
Application: Panography
Iterative algorithms
Robust least squares and RANSAC
3D alignment
Pose estimation
Linear algorithms
Iterative algorithms
Application: Augmented reality
Geometric intrinsic calibration
Calibration patterns
Vanishing points
Application: Single view metrology
Rotational motion
Radial distortion
11
Structure from motion
Triangulation
Two-frame structure from motion
Projective (uncalibrated)
reconstruction
Self-calibration
Application: View morphing
Factorization
Perspective and projective
factorization
Application: Sparse 3D model
extraction
Bundle adjustment
Exploiting sparsity
Application: Match move and
augmented reality
Uncertainty and ambiguities
Application: Reconstruction from
Internet photos
Constrained structure and motion
Line-based techniques
Plane-based techniques
12
Dense motion estimation
Translational alignment
Hierarchical motion estimation
Fourier-based alignment
Incremental refinement
Parametric motion
Application: Video stabilization
Learned motion models
Spline-based motion
Application: Medical image
registration
Optical flow
Multi-frame motion estimation
Application: Video denoising
Application: De-interlacing
Layered motion
Application: Frame
interpolation
Transparent layers and
reflections
13
Image Stitching
Motion models
Planar perspective motion
Application: Whiteboard and
document scanning
Rotational panoramas
Gap closing
Application: Video summarization
and compression
Cylindrical and spherical coordinates
Global alignment
Bundle adjustment
Parallax removal
Recognizing panoramas
Direct vs. feature-based alignment
Compositing
Choosing a compositing surface
Pixel selection and weighting (de-
ghosting)
Application: Photomontage
Blending
14
Computational Photography
Photometric calibration
Radiometric response function
Noise level estimation
Vignetting
Optical blur (spatial response)
estimation
High dynamic range imaging
Tone mapping
Application: Flash photography
Super-resolution and blur removal
Color image demosaicing
Application: Colorization
Image matting and compositing
Blue screen matting
Natural image matting
Optimization-based matting
Smoke, shadow, and flash matting
Video matting
Texture analysis and synthesis
Application: Hole filling and
inpainting
Application: Non-photorealistic
rendering 15
Stereo correspondence
Epipolar geometry
Rectification
Plane sweep
Sparse correspondence
3D curves and profiles
Dense correspondence
Similarity measures
Local methods
Sub-pixel estimation and uncertainty
Application: Stereo-based head
tracking
Global optimization
Dynamic programming
Segmentation-based techniques
Application: Z-keying and
background replacement
Multi-view stereo
Volumetric and 3D surface
reconstruction
Shape from silhouettes
16
3D reconstruction
Shape from X
Shape from shading and photometric stereo
Shape from texture
Shape from focus
Active rangefinding
Range data merging
Application: Digital heritage
Surface representations
Surface interpolation
Surface simplification
Geometry images
Point-based representations
Volumetric representations
Implicit surfaces and level sets
Model-based reconstruction
Architecture
Heads and faces
Application: Facial animation
Whole body modeling and tracking
Recovering texture maps and albedos
Estimating BRDFs
Application: 3D photography
17
Image-based rendering
View interpolation
View-dependent texture maps
Application: Photo Tourism
Layered depth images
Impostors, sprites, and layers
Light fields and Lumigraphs
Unstructured Lumigraph
Surface light fields
Application: Concentric mosaics
Environment mattes
Higher-dimensional light fields
The modeling to rendering
continuum
Video-based rendering
Video-based animation
Video textures
Application: Animating pictures
3D Video
Application: Video-based
walkthroughs
18
Object detection
Recognition
Face detection
Pedestrian detection
Face recognition
Eigenfaces
Active appearance and 3D shape models
Application: Personal photo collections
Instance recognition
Geometric alignment
Large databases
Application: Location recognition
Category recognition
Bag of words
Part-based models
Recognition with segmentation
Application: Intelligent photo editing
Context and scene understanding
Learning and large image collections
Application: Image search
Recognition databases and test sets
19
Computer Vision Levels
Low-level vision (early vision)
Image formation
Edge detection & image filtering
Optical flow
Segmentation
Shape matching
Stereopsis
Mid-level vision
Object tracking
Human motion analysis
High-level vision
Object recognition
Event detection
Scene & video understanding
20
Computer Vision Algorithms
Structure from motion algorithms can reconstruct a sparse 3D
point model of a large complex scene from hundreds of partially
overlapping photographs(Snavely, Seitz, and Szeliski 2006)c
2006 ACM.
21
Computer Vision Algorithms
Stereo matching algorithms can build a detailed 3D model of a
building from hundreds of differently exposed photographs
taken from the Internet (Goesele, Snavely, Curless et al. 2007) c
2007 IEEE.
22
Computer Vision Algorithms
Person tracking algorithms can track a person walking in front
of a cluttered background (Sidenbladh, Black, and Fleet 2000) c
2000 Springer
23
Computer Vision Algorithms
Face detection algorithms, coupled with color-based clothing
and hair detection algorithms, can locate and recognize the
individuals in this image (Sivic, Zitnick, and Szeliski 2006) c
2006 Springer.
24
Computer Vision Applications
Optical character recognition (OCR): reading handwritten
postal codes on letters (Figure 1.4a) and automatic number plate
recognition (ANPR)
25
Computer Vision Applications
Machine inspection: rapid parts inspection for quality assurance
using stereo vision with specialized illumination to measure
tolerances on aircraft wings or auto body parts or looking for
defects in steel castings using X-ray vision;
26
Computer Vision Applications
Retail: object recognition for automated checkout
27
Computer Vision Applications
Medical imaging: registering pre-operative and intra-operative
imagery or performing long-term studies of people’s brain
morphology as they
28
Computer Vision Applications
Automotive safety: detecting unexpected obstacles such as
pedestrians on the street, under conditions where active vision
techniques such as radar or lidar do not work well
29
Computer Vision Applications
Surveillance: monitoring for intruders, analyzing highway
traffic, and monitoring pools for drowning victims;
30
Computer Vision Application
Fingerprint recognition and biometrics: for automatic access
authentication as well
31
Computer Vision Applications
32
Computer Vision Applications
33
Computer Vision Applications
34
Computer Vision Applications
35
A Brief History
36
A Brief History
37
A Brief History
38
A Brief History
39
A Brief History
40