Computer Vision

Eduardo Avendaño Fernández

Hello!
I am Eduardo Avendaño Fernandez

I am here because I like DSP

and Computer Vision.
You can find me at:
@eduardoavendanofernandez

2
Introduction to
Artificial Vision
Let’s start with the first set of slides

3
“

4 Magic Leap, Oculus, Hololens, etc.

Every image tells a story

◎ Goal of computer vision:

perceive the “story” behind
the picture
◎ Compute properties of the
world
○ 3D shape
○ Names of people or objects
○ What happened?
5
The goal of computer vision

6
But humans can tell a lot about a scene from a little
information…

Source: “80 million tiny images” by Torralba, et al. 7

Can computers match human perception?
Credits: Fei-Fei, Fergus & Torralba

Sky
Building

Flag
Street Lamp

Banner Wall

Bus

Bus Cars 8
The Goal of Computer Vision

◎ Forensics

Source: Nayar and Nishino, “Eyes for Relighting” 9

Source: Nayar and Nishino, “Eyes for Relighting” 10
The Goal of Computer Vision

Source: Nayar and Nishino, “Eyes for

Relighting”

11
 Improve photos (“Computational Photography”)

Super-resolution (source: 2d3)

Low-light photography
(credit: Hasinoff et al., SIGGRAPH ASIA 2016)

Inpainting / image completion

Depth of field on cell phone camera (image credit: Hays and Efros)
(source: Google Research Blog) 12
Source: Nayar and Nishino, “Eyes for Relighting”
 ◎ study computer vision?
Why
◎ Billions of images/videos captured per day

◎ Huge number of potential applications

◎ The next slides show the current state of the art 13
 Optical character recognition (OCR)

Digit recognition, AT&T labs (1990’s) License plate readers

http://en.wikipedia.org/wiki/Automatic_number_plate_recognition
http://yann.lecun.com/exdb/lenet/

Sudoku grabber http://sudokugrab.blogspot.com/ Automatic check processing 14

Face analysis and
recognition

15
Vision-based biometrics

◎ Who is she?
“How the Afghan Girl was Identified by Her Iris Patterns” Read
the story

Source: S. Seitz 16
Login without a password

Fingerprint scanners on Face unlock on Apple iPhone X

many new smartphones See also http://www.sensiblevision.com/
and other devices
17
Special effects: shape capture

The Matrix movies, ESC Entertainment, XYZRGB, NRC

Source: S. Seitz

18
Special effects:
motion capture

Source:
S. Seitz

Pirates of the Carribean, Industrial Light and Magic 19

20
Which face is real?

https://www.whichfaceisreal.com/ 21
Image synthesis

Zhu, et al., Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, ICCV 2017
22
 Smart Cars

◎ Mobileye
◎ Tesla Autopilot
◎ Safety features

Pedestrian collision warning

Forward collision warning
Lane departure warning
Headway monitoring and warning

23
Vision-based interaction: Xbox Kinect

24
Computer vision research in biology

http://www.vision.caltech.edu/visipedia/
http://leafsnap.com/
25
 Applications: 3D Scanning

Scanning Michelangelo’s “The David”

• UW Prof. Brian Curless, collaborator
• The Digital Michelangelo Project • 2 BILLION polygons, accuracy to .29mm
- http://graphics.stanford.edu/projects/mich/
26
Applications: 3D Scanning

27
Applications: 3D Scanning

28
Self-driven cars

◎ Waymo

29
Robotics

Amazon Prime Air

Amazon Picking Challenge

http://www.robocup2016.org/en/events/amazon-picking-challenge/

NASA’s Mars Curiosity Rover

30
https://en.wikipedia.org/wiki/Curiosity_(rover) Amazon Scout
 Medical imaging

3D imaging (MRI, CT)

Skin cancer classification with deep learning https://cs.stanford.edu/people/esteva/nature/

31
32
Virtual & Augmented Reality

Hand & body

6DoF head tracking
tracking

3D scene understanding 3D-360 video capture 33

Current state
of the art
Many examples < 5 years old
Active research area (ML, DL…)
Startups (robotics, autonomous
vehicles, medical imaging,
construction, inspection, VR/AR)
34
Why computer vision matters

Safety Health Security

Comfort Fun Access 35

Ridiculously brief history of computer vision

◎ 1966: Minsky assigns computer vision as an undergrad summer project

◎ 1960’s: interpretation of synthetic worlds
◎ 1970’s: some progress on interpreting selected images
◎ 1980’s: ANNs come and go; shift toward geometry and increased
mathematical rigor
◎ 1990’s: face recognition; statistical analysis in vogue (Michael Jackson)
◎ 2000’s: broader recognition; large annotated datasets available; video
processing starts
◎ 2010’s: Deep learning with ConvNets
◎ 2020’s: Widespread autonomous vehicles?
◎ 2030’s: robot uprising? 36
Why is computer vision difficult?

Viewpoint variation

Illumination Credit: Flickr user michaelpaul

Scale 37
Why is computer vision difficult?

Motion (Source: S. Lazebnik)

Intra-class variation

38

Background clutter Occlusion

Bottom line

◎ Perception is an inherently ambiguous problem

○ Many different 3D scenes could have given rise to a
given 2D image

◎ We often must use prior knowledge about the world’s

structure

Artist Julian Beever with his anamorphic Coke bottle 39

Course overview

1. Low-level vision
○ image processing, edge detection,
feature detection, cameras, image
formation
2. Geometry and algorithms
○ projective geometry, stereo,
structure from motion, optimization
3. Recognition
○ face detection / recognition,
category recognition, segmentation

40
1. Low-level vision: Basic image processing and image
formation

* =
Filtering, edge detection

Image formation 41
Feature extraction
Geometry

Image credit: IDS Imaging

Projective geometry
Stereo vision

Multi-view stereo Structure from motion 42

Recognition
“dog”

Image classification
Object detection

Convolutional Neural Networks 43

Integridad Académica

◎ Assignments will be done solo or in pairs

◎ Exams: theoretical and practical on weeks 3, 6, 9 and
15
◎ Delays on the delivery at Moodle will be penalized
(first 6 hours -0.5 Units, 12 hours -1 Unit, 24 hours - 2
Unit)

44
What is Computer Vision?

◎ Input: images or video

◎ Output: description of the world
○ Many levels of description

45
Low-Level or “Early” Vision

Considers local
properties of an image

“There’s an edge!”

46
Mid-Level Vision

Grouping and
segmentation

“There’s an object
and a background!”

47
High-Level Vision

Recognition

“It’s a chair!” 48
Vision and Other Fields

Cognitive Psychology Signal Processing

Computer Vision
Computer Graphics

Pattern Analysis

Metrology
49
 Questions?

[email protected]