1807 ESIEA ComputerVision-1
1807 ESIEA ComputerVision-1
Emma Beauxis-Aussalet
18-07-2019
INTRODUCTION
TO COMPUTER
VISION
INTRODUCTION TO CV
WHAT IS
COMPUTER VISION?
3
HAVE YOU EVER USED IT?
Most probably…
Source: https://cs.brown.edu/courses/csci1430/lectures/2019Spring_01_Introduction.pdf
DEFINITION
Attributed to The British Machine Vision Association and Society for Pattern Recognition
5
MULTIDISCIPLINARY
Psychology Machine
Learning
Neuroscience Robotics
Camera Scanner Sensor
6
SUB-DOMAINS
IMAGE
(PRE-)PROCESSING
8
PIXEL (picture element)
9
PIXEL (picture element)
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 0 0 0 0 180 180 180 180 180 180 180 180 0 0 0 0 255 255 255 255 255 255 255 255
255 255 255 255 0 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 0 0 255 255 255 255 255 255
255 255 255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 0 255 255 255 255 255
255 255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 0 255 255 255 255
255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 0 255 255 255
255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 0 255 255 255
0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 0 255 255
0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 0 255 255
0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 0 255 255
0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 0 255 255
0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 0 255 255
255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 0 255 255 255
255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 90 0 255 255 255
255 255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 90 90 0 255 255 255 255
255 255 255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 90 90 90 90 90 90 0 255 255 255 255
255 255 255 255 255 255 0 0 0 0 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0 255 255 255 255
255 255 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 255 255 255
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 0 90 90 90 90 0 0 255
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 0 90 90 90 90 0
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 0 0 0 255
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255
10
PIXEL (picture element)
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 255 255 255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 0 0 0 0 180 180 180 180 180 180 180 180 0 0 0 0 255 255 255 255 255 255 255 255
255 255 255 255 0 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 0 0 255 255 255 255 255 255
255 255 255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 0 255 255 255 255 255
255 255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 0 255 255 255 255
255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 0 255 255 255
255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 0 255 255 255
0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 0 255 255
0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 0 255 255
0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 0 255 255
0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 0 255 255
0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 0 255 255
255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 0 255 255 255
255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 90 0 255 255 255
255 255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 90 90 0 255 255 255 255
255 255 255 0 180 180 180 180 180 180 180 180 180 180 180 180 180 90 90 90 90 90 90 90 90 0 255 255 255 255
255 255 255 255 255 255 0 0 0 0 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0 255 255 255 255
255 255 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 255 255 255
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 0 90 90 90 90 0 0 255
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 0 90 90 90 90 0
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 0 0 0 0 255
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255
11
PIXEL (picture element)
3 colors…
12
PIXEL (picture element)
3 colors… …3 channels
13
(PRE-)PROCESSING
‣ BRIGHTNESS, CONTRAST
‣ HISTOGRAM EQUALISATION
‣ COLOR NORMALIZATION
‣ FILTERING
14
Source: http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf
BRIGHTNESS, CONTRAST
Add X to
all pixel values… …to increase
.brightness.
Multiply X with
all pixel values… …to increase
.contrast.
Source: http://mccormickml.com/2013/05/09/hog-person-detector-tutorial/ 15
BRIGHTNESS, CONTRAST
# pixel
less
brightness
Source: https://en.wikipedia.org/wiki/Histogram_equalization 18
COLOR NORMALISATION
Source: http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf 19
FILTERING
Source: https://cs.brown.edu/courses/csci1430/lectures/2019Spring_01_Introduction.pdf
20
FILTERING
21
FILTERING
180 90 0
180 90 0
180 90 0
0 x 180 + 0 x 90 + 0 x 0
+ 0 x 180 + 0 x 90 + 1 x 0
+ 0 x 180 + 0 x 90 + 0 x 0
=0
22
FILTERING
180 90 0
180 0 0
180 90 0
0 x 180 + 0 x 90 + 0 x 0
+ 0 x 180 + 0 x 90 + 1 x 0
+ 0 x 180 + 0 x 90 + 0 x 0
=0
23
FILTERING
90 0 255
90 0 255
90 0 255
0 x 90 + 0 x 0 + 0 x 255
+ 0 x 90 + 0 x 0 + 1 x 255
+ 0 x 90 + 0 x 0 + 0 x 255
= 255
24
FILTERING
90 0 255
90 255 255
90 0 255
0 x 90 + 0 x 0 + 0 x 255
+ 0 x 90 + 0 x 0 + 1 x 255
+ 0 x 90 + 0 x 0 + 0 x 255
= 255
25
FILTERING
180 90 0 255
180 90 0 255
26
FILTERING
27
FILTERING
Source: https://cs.brown.edu/courses/csci1430/lectures/2019Spring_01_Introduction.pdf
28
FILTERING
Source: https://cs.brown.edu/courses/csci1430/lectures/2019Spring_01_Introduction.pdf
29
FILTERING
Source: https://en.wikipedia.org/wiki/Kernel_%28image_processing%29
30
INTRODUCTION TO CV
FEATURE DETECTION
INTEREST POINTS
Primarily to
match images
Source: http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf 32
EDGE
Source: https://en.wikipedia.org/wiki/Edge_detection
33
EDGE
easy
more
ambiguous
34
DOG (DIFFERENCE OF GAUSSIAN)
180 90 0
180 90 0
180 90 0
35
DOG (DIFFERENCE OF GAUSSIAN)
180 90 90 0 255
36
DOG (DIFFERENCE OF GAUSSIAN)
1/16 * ( 1 * 180 + 2 * 90 + 1 * 0
+ 2 * 180 + 4 * 90 + 2 * 0 180 180 0 255 255
37
DOG (DIFFERENCE OF GAUSSIAN)
Source: https://en.wikipedia.org/wiki/Difference_of_Gaussians
38
HOG (HISTOGRAM OF ORIENTED GRADIENTS)
Source: http://mccormickml.com/2013/05/07/gradient-vectors/ 39
HOG (HISTOGRAM OF ORIENTED GRADIENTS)
Source: https://www.learnopencv.com/histogram-of-oriented-gradients/ 40
HOG (HISTOGRAM OF ORIENTED GRADIENTS)
Source: https://www.learnopencv.com/histogram-of-oriented-gradients/ 41
HOG (HISTOGRAM OF ORIENTED GRADIENTS)
Source: https://www.learnopencv.com/histogram-of-oriented-gradients/ 42
HOG (HISTOGRAM OF ORIENTED GRADIENTS)
43
HOG (HISTOGRAM OF ORIENTED GRADIENTS)
‣ Scale
‣ Rotation
‣ Illumination
45
SIFT (SCALE-INVARIANT FEATURE TRANSFORM)
Source: http://aishack.in/tutorials/sift-scale-invariant-feature-transform-introduction/
46
SIFT (SCALE-INVARIANT FEATURE TRANSFORM)
Source: http://aishack.in/tutorials/sift-scale-invariant-feature-transform-introduction/
47
SIFT (SCALE-INVARIANT FEATURE TRANSFORM)
Source: http://aishack.in/tutorials/sift-scale-invariant-feature-transform-introduction/
48
SIFT (SCALE-INVARIANT FEATURE TRANSFORM)
Source: http://aishack.in/tutorials/sift-scale-invariant-feature-transform-introduction/
49
SIFT (SCALE-INVARIANT FEATURE TRANSFORM)
Source: http://aishack.in/tutorials/sift-scale-invariant-feature-transform-introduction/
50
SIFT (SCALE-INVARIANT FEATURE TRANSFORM)
Find the bin with max #pixels,
select all bins with #pixels > 80% max #pixels.
Source: http://aishack.in/tutorials/sift-scale-invariant-feature-transform-introduction/
52
SIFT (SCALE-INVARIANT FEATURE TRANSFORM)
Source: Tony Lindeberg. Image Matching Using Generalized Scale-Space Interest Points. https://people.kth.se/~tony/papers/Lin15-JMIV.pdf 53
INTRODUCTION TO CV
SEGMENTATION
WHAT IS SEGMENTATION?
Source: http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf 55
SEGMENTATION TECHNIQUES
Source: https://www.slideshare.net/lalitxp/image-texture-analysis?next_slideshow=1 56
WITHOUT CLASSIFICATION
Source: http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf 57
UNSUPERVISED CLASSIFICATION
Clustering & neural networks* can be used to group similar patches.
* e.g., self-organizing maps (SOM) or constraint satisfaction neural networks (CSNN)
Source: http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf 58
UNSUPERVISED CLASSIFICATION
Clustering & neural networks* can be used to group similar patches.
* e.g., self-organizing maps (SOM) or constraint satisfaction neural networks (CSNN)
Source: https://www.slideshare.net/lalitxp/image-texture-analysis?next_slideshow=1 59
UNSUPERVISED CLASSIFICATION
Clustering & neural networks* can be used to group similar patches.
* e.g., self-organizing maps (SOM) or constraint satisfaction neural networks (CSNN)
Source: https://www.slideshare.net/lalitxp/image-texture-analysis?next_slideshow=1 60
SUPERVISED CLASSIFICATION
Classification
Model
Source: http://people.ee.ethz.ch/~cattin/MIA-ETH/pdf/MIA-08-SupervisedSegmentation.pdf 61
PROBABILISTIC SEGMENTATION
Source: https://www.slideshare.net/lalitxp/image-texture-analysis?next_slideshow=1 62
RESOURCES
Title of Resource:
Resource Type: Website
Description: A definition and discussion of the principles of grouping.
Title of Resource:
Resource Type: Video (4:26)
Description: A Khan Academy lecture on bottom-up versus top-down processing.
Title of Resource: Thresholding
Resource Type: Website
Description: Thresholding is the simplest method of image segmentation. From a grayscale image, thresholding can be used to create binary images.
Title of Resource: Otsu Thresholding
Resource Type: Website
Description: Converting a greyscale image to monochrome is a common image processing task. Otsu's method, named after its inventor Nobuyuki Otsu, is one of many binarization algorithms. This page describes how the
algorithm works and provides a Java implementation, which can be easily ported to other languages.
Title of Resource:
Resource Type: PDF
Description: This paper reviews the main approaches of partitioning an image into regions by using gray values in order to reach a correct interpretation of the image.
Title of Resource: Cluster Analysis
Resource Type: Website
Description: The definition of cluster analysis.
Title of Resource:
Resource Type: Video (1:21)
Description: The video shows my K-Means Clustering algorithm running on an image, iterating from K=1 to K=80 clusters, with the last 3 frames being the original image.
Title of Resource:
Resource Type: Video (2:58)
Description: This video accompanies the article "Semantic Soft Segmentation" by Yağız Aksoy, Tae-Hyun Oh, Sylvain Paris, Marc Pollefeys and Wojciech Matusik. The article and additional resources are available on the
project webpage: https://yaksoy.github.io/sss/.
Title of Resource:
Resource Type: Website
Description: How graph cuts are applied in the field of computer vision.
Title of Resource:
Resource Type: Video (2:11)
Description: This video is part of the Udacity course "Introduction to Computer Vision"
63
INTRODUCTION TO CV
OBJECT RECOGNITION
CLASSIFICATION TASKS
65
TYPICAL PIPELINE
Source: https://www.learnopencv.com/image-recognition-and-object-detection-part1/ 66
TYPICAL PIPELINE
‣ Neural Networks & Deep Learning are the most common technique.
‣ Support Vector Machines (SVM) learn boundaries between classes
in the chosen feature space.
‣ Boosting combines weak (extremely simple) classifiers.
Source: http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf 69
TECHNIQUES
‣ Neural Networks & Deep Learning are the most common technique.
‣ Support Vector Machines (SVM) learn boundaries between classes
in the chosen feature space.
‣ Boosting combines weak (extremely simple) classifiers.
‣ Bag-of-Word learns typical parts of objects and their spatial distribution.
‣ Active Appearance Models specify typical key points, their spatial
distribution, and their variability.
Source: http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf
70
RESOURCES
Title of Resource:
Resource Type: Video
Description: Find meaning in visual data on Watson Studio! Analyze images for scenes, objects, faces, and other custom content. Take advantage of pretrained models, or create your
own custom classifier. Develop smart applications that analyze the visual content of images or video frames to understand what is happening in a scene.Then, deploy your model for
use in applications.
Title of Resource:
Resource Type: Website Article
Description: Compares and contrasts Computer Vision and Visual Recognition to clearly explain their differences.
Title of Resource: Hough Transform
Resource Type: Website
Description: Defines a Hough Transform.
Title of Resource:
Resource Type: Video
Description: This video explains how the Hough Transform works to detect lines in images. First, apply an edge detection algorithm to the input image, and then compute the Hough
Transform to find the combination of Rho and Theta values in which there are more occurrences of lines.
Title of Resource:
Resource Type: Journal Article (PDF)
Description: This paper introduces a new scene-centric database called Places with over 7 million labeled pictures of scenes. It then proposes new methods to compare the density
and diversity of image datasets and shows that Places is as dense as other scene datasets and has more diversity.
Title of Resource:
Resource Type: Website
Description: Defines k-nearest neighbors algorithm.
71
DIGITAL SOCIETY SCHOOL
Emma Beauxis-Aussalet
18-07-2019
QUESTIONS /
DISCUSSIONS
YOU
TRANSFORM
SOCIETY
BY DESIGN
DIGITAL SOCIETY SCHOOL
WHERE CHANGE TAKES
SHAPE