0% found this document useful (0 votes)
237 views

Lecture 01 Introduction To Computer Vision PDF

Uploaded by

Sai Sukruth Goud
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
237 views

Lecture 01 Introduction To Computer Vision PDF

Uploaded by

Sai Sukruth Goud
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 118

COMP499G/691G

COMPUTER VISION

LECTURE 1:
INTRODUCTION TO COMPUTER VISION

Copyright © Charalambos Poullis


Today’s Lecture
• Introductions
• What is Computer Vision?
• Computer Vision @ Concordia University
• Course administrative details
• Questions

Copyright © Charalambos Poullis


Teaching Assistant/Grader
TAs: Amin Karimi, Farhan Rahman Wasee
Grader: Qiao [Jocelyn] Chen
Office: EV3.260
Office hours: During lab sessions or by appointment

Copyright © Charalambos Poullis


Instructor
Dr. Charalambos [Charis] Poullis

Associate Professor in Computer Vision


Office: EV3.183
Office hours: 2-3pm every Monday
or by appointment

Copyright © Charalambos Poullis


Copyright © Charalambos Poullis
CYPRUS

Copyright © Charalambos Poullis


CYPRUS

Copyright © Charalambos Poullis


Copyright © Charalambos Poullis
CYPRUS

Copyright © Charalambos Poullis


CYPRUS

Copyright © Charalambos Poullis


CYPRUS

Copyright © Charalambos Poullis


Copyright © Charalambos Poullis
Research Interests
• Fundamental research

Computer Vision

Computer Graphics
• Applied Research

Virtual Reality

Augmented Reality

www.theICTlab.org
Copyright © Charalambos Poullis
Recent work

Semantic Segmentation from Remote Sensor Data and the Exploitation of Latent Learning for Classification of Auxiliary
Tasks, B. Chaterjee, C. Poullis, CRV 2019

Copyright © Charalambos Poullis


Recent work

Delineation of Road Networks Using Deep Residual Neural Networks and Iterative Hough Transform, P. Xu, C. Poullis, ISVC
2019

Copyright © Charalambos Poullis


Recent work

Inattentional Blindness for Redirected Walking Using Dynamic Foveated Rendering, Under Review 2020

Copyright © Charalambos Poullis


The goal of computer vision
• To extract “meaning” from pixels

What we see
Source: S. Narasimhan
The goal of computer vision
• To extract “meaning” from pixels

What we see What a computer sees


Source: S. Narasimhan
The goal of computer vision
• To extract “meaning” from pixels

Humans are remarkably good at this…


Source: “80 million tiny images” by Torralba et al.
Can computers match (or beat) human
vision?
Can computers match (or beat) human
vision?
• Yes and no (but mostly no!)
• humans are much better at
“hard” things
• computers can be better at
“easy” things
Human perception has its shortcomings…
What kind of information can be
extracted from an image?
What kind of information can be
extracted from an image?

Semantic information
What kind of information can be
extracted from an image?

European
City

Outdoor scene

Semantic information
What kind of information can be
extracted from an image?
tree tree
roof
sky chimney
building
building
window
door
trashcan car car
person
Outdoor scene
ground City European

Semantic information
What kind of information can be
extracted from an image?
tree tree
roof
sky chimney
building
building
window
door
trashcan car car
person
Outdoor scene
ground City European

Semantic information Geometric information


Why study computer vision?
Why study computer vision?
• Vision is useful
Why study computer vision?
• Vision is useful
• Vision is interesting
Why study computer vision?
• Vision is useful
• Vision is interesting
• Vision is difficult
• Half of primate cerebral cortex is devoted to visual
processing
• Achieving human-level visual perception is probably
“AI-complete”
Why computer vision matters

Safety Health Security

Comfort Fun Access


Ridiculously brief history of computer vision
• 1966: Minsky assigns computer vision as an
undergrad summer project
Ridiculously brief history of computer vision
• 1966: Minsky assigns computer vision as an
undergrad summer project
–“spend the summer linking a camera to a computer and getting the
computer to describe what it saw”
Ridiculously brief history of computer vision
• 1966: Minsky assigns computer vision as an
undergrad summer project
–“spend the summer linking a camera to a computer and getting the
computer to describe what it saw”
• 1960’s: interpretation of synthetic worlds Guzman ‘68
Ridiculously brief history of computer vision
• 1966: Minsky assigns computer vision as an
undergrad summer project
–“spend the summer linking a camera to a computer and getting the
computer to describe what it saw”
• 1960’s: interpretation of synthetic worlds Guzman ‘68
• 1970’s: some progress on interpreting selected
images

Ohta Kanade ‘78


Ridiculously brief history of computer vision
• 1966: Minsky assigns computer vision as an
undergrad summer project
–“spend the summer linking a camera to a computer and getting the
computer to describe what it saw”
• 1960’s: interpretation of synthetic worlds Guzman ‘68
• 1970’s: some progress on interpreting selected
images
• 1980’s: ANNs come and go; shift toward geometry
and increased mathematical rigor
Ohta Kanade ‘78
Ridiculously brief history of computer vision
• 1966: Minsky assigns computer vision as an
undergrad summer project
–“spend the summer linking a camera to a computer and getting the
computer to describe what it saw”
• 1960’s: interpretation of synthetic worlds Guzman ‘68
• 1970’s: some progress on interpreting selected
images
• 1980’s: ANNs come and go; shift toward geometry
and increased mathematical rigor
• 1990’s: face recognition; statistical analysis in vogue Ohta Kanade ‘78

Turk and Pentland ‘91


Ridiculously brief history of computer vision
• 1966: Minsky assigns computer vision as an
undergrad summer project
–“spend the summer linking a camera to a computer and getting the
computer to describe what it saw”
• 1960’s: interpretation of synthetic worlds Guzman ‘68
• 1970’s: some progress on interpreting selected
images
• 1980’s: ANNs come and go; shift toward geometry
and increased mathematical rigor
• 1990’s: face recognition; statistical analysis in vogue Ohta Kanade ‘78
• 2000’s: broader recognition; large annotated
datasets available; video processing starts

Turk and Pentland ‘91


Ridiculously brief history of computer vision
• 1966: Minsky assigns computer vision as an
undergrad summer project
–“spend the summer linking a camera to a computer and getting the
computer to describe what it saw”
• 1960’s: interpretation of synthetic worlds Guzman ‘68
• 1970’s: some progress on interpreting selected
images
• 1980’s: ANNs come and go; shift toward geometry
and increased mathematical rigor
• 1990’s: face recognition; statistical analysis in vogue Ohta Kanade ‘78
• 2000’s: broader recognition; large annotated
datasets available; video processing starts
• 2010’s: Deep learning with ConvNets

Turk and Pentland ‘91


Ridiculously brief history of computer vision
• 1966: Minsky assigns computer vision as an
undergrad summer project
–“spend the summer linking a camera to a computer and getting the
computer to describe what it saw”
• 1960’s: interpretation of synthetic worlds Guzman ‘68
• 1970’s: some progress on interpreting selected
images
• 1980’s: ANNs come and go; shift toward geometry
and increased mathematical rigor
• 1990’s: face recognition; statistical analysis in vogue Ohta Kanade ‘78
• 2000’s: broader recognition; large annotated
datasets available; video processing starts
• 2010’s: Deep learning with ConvNets
• 2030’s: robot uprising?
Turk and Pentland ‘91
Origins of computer vision

L. G. Roberts,
Machine Perception of Three Dime
nsional Solids
, Ph.D. thesis, MIT Department of
Electrical Engineering, 1963.
Successes of computer vision to date
Optical character recognition (OCR)

Digit recognition
yann.lecun.com

Source: S. Seitz, N. Snavely


Optical character recognition (OCR)

Digit recognition License plate readers


yann.lecun.com http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Source: S. Seitz, N. Snavely


Optical character recognition (OCR)

Digit recognition License plate readers


yann.lecun.com http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Automatic check processing Source: S. Seitz, N. Snavely


Optical character recognition (OCR)

Digit recognition License plate readers


yann.lecun.com http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Sudoku grabber
http://sudokugrab.blogspot.com/

https://www.youtube.com/watch?v=oI
mMJ6p6mKE
Automatic check processing Source: S. Seitz, N. Snavely
Biometrics

Face recognition systems now


Fingerprint scanners on
beginning to appear more widely
many new laptops,
http://www.sensiblevision.com/
other devices

Source: S. Seitz
Biometrics

How the Afghan Girl was Identified by Her Iris Patterns

Source: S. Seitz
Face detection

Many consumer digital cameras now detect faces

Source: S. Seitz
Smile detection

Sony Cyber-shot® T70 Digital Still Camera Source: S. Seitz


Face detection for privacy protection
Technology gone wild…
Technology gone wild…
Technology gone wild…
Technology gone wild…
Face recognition: Apple iPhoto software

http://www.apple.com/ilife/iphoto/
Visual search: Google search by image
Google self-driving cars

https://www.youtube.com/watch?v=bDOnn0-4Nq8

• Google’s self-driving car passes 300,000 miles (Forbes, 8/15/2012)


• Nissan pledges affordable self-driving car models by 2020
(CNET, 8/27/2013)
Automotive safety

Mobileye: Vision systems in high-end BMW, GM, Volvo models


• Pedestrian collision warning https://www.youtube.com/watch?v=HXpiyLUEOOY
• Forward collision warning
• Lane departure warning
• Headway monitoring and warning Source: A. Shashua, S. Seitz
Vision-based interaction: Xbox Kinect
3D Reconstruction: Kinect Fusion

YouTube Video
Occipital: 3D structure sensor for iPad

http://www.kickstarter.com/projects/occi
pital/structure-sensor-capture-the-world
-in-3d

http://structure.io/#home-about-vid
3D reconstruction from photo collections

Q. Shan, R. Adams, B. Curless, Y. Furukawa, and S. Seitz,


The Visual Turing Test for Scene Reconstruction, 3DV 2013
YouTube Video
Object recognition
Google Goggles
Bing Vision
Special effects: shape and motion capture

Source: S. Seitz
Vision in space

NASA'S Mars Exploration Rover Spirit captured this westward view from atop
a low plateau where Spirit spent the closing months of 2007.

Vision systems (JPL) used for several tasks


• Panorama stitching
• 3D terrain modeling
• Obstacle detection, position tracking
• For more, read “Computer Vision on Mars” by Matthies et al.
Industrial robots

Vision-guided robots position nut runners on wheels


Mobile robots

NASA’s Mars Spirit Rover


http://www.robocup.org/
http://en.wikipedia.org/wiki/Spirit_rover

Saxena et al. 2008


STAIR at Stanford
Medical imaging

3D imaging Image guided surgery


MRI, CT Grimson et al., MIT
Why is computer vision difficult?
Challenges: viewpoint variation

Michelangelo 1475-1564 slide credit: Fei-Fei, Fergus & Torralba


Challenges: viewpoint variation

slide credit: Fei Fei Li, Andrej Karpathy, Justin Johnson


Challenges: illumination

image credit: J. Koenderink


Challenges: illumination

slide credit: Fei Fei Li, Andrej Karpathy, Justin Johnson


Challenges: scale

slide credit: Fei-Fei, Fergus & Torralba


Challenges: deformation

Xu, Beihong 1943


slide credit: Fei-Fei, Fergus & Torralba
Challenges: deformation

slide credit: Fei Fei Li, Andrej Karpathy, Justin Johnson


Challenges: object intra-class variation

slide credit: Fei-Fei, Fergus & Torralba


Challenges: object intra-class variation

slide credit: Fei Fei Li, Andrej Karpathy, Justin Johnson


Challenges: occlusion

slide credit: Fei Fei Li, Andrej Karpathy, Justin Johnson


Challenges:clutter

slide credit: Fei Fei Li, Andrej Karpathy, Justin Johnson


Challenges: occlusion, clutter

Image source: National Geographic


Challenges: Motion
Challenges: ambiguity

Source: Rob Fergus and Antonio Torralba


Challenges: ambiguity

Source: Rob Fergus and Antonio Torralba


Challenges: ambiguity

slide credit: Fei-Fei, Fergus & Torralba


Challenges: ambiguity
• Many different 3D scenes could have given rise to a
particular 2D picture
Review: Intro to computer vision
• State-of-the-art applications
• Challenges of vision
– Viewpoint and lighting variation
– Intra-class variations: size, shape, deformation, etc.
– Nuisances: motion, blur, noise, etc.
– Intrinsic ambiguity
Challenges or opportunities?
• Images are confusing, but they also reveal the structure of
the world through numerous cues
• Our job is to interpret the cues!
Depth cues: Linear perspective
Depth cues: Parallax
Depth cues: Parallax
Shape cues: Texture gradient
Shape and lighting cues: Shading
Grouping cues: Similarity (color, texture,
proximity)
Grouping cues: “Common fate”

Image credit: Arthus-Bertrand (via F. Durand)


The computer vision industry
• A list of companies here:

http://www.cs.ubc.ca/spider/lowe/vision.html
Robotics
Computer Vision

Machine
Scope of Human Computer
Interaction
Learning COMP 425/6341
Image Processing Medical Imaging
Graphics
Feature Matching
Computational Recognition
Neuroscience
Photography

Optics
Computer Vision and Nearby Fields
• Computer Graphics: Models to Images
• Comp. Photography: Images to Images
• Computer Vision: Images to Models
Course Website
http://www.poullis.org/courses/2020/Winter/COMP425-6341

▪lecture notes are posted the day of the class


▪bring to class and keep notes

▪The lecture notes are password protected


▪user: computer_vision
▪pass: lecture_notes2020

Copyright © Charalambos Poullis


Syllabus - Course Outline
Posted on the website

Copyright © Charalambos Poullis


Pre-requisites
• Data structures
• A good working knowledge of C and C++
programming
• Linear algebra
• Vector Calculus

• No prior knowledge of image processing, computer


vision is assumed

Copyright © Charalambos Poullis


Grading
• Programming assignments (10%+15%)
• Project (20%)
• Quiz 1 (25%)
• Quiz 2 (30%)

• Late submission policy


• Extra 3 days at the cost of 2 points [out of 10]
• No extra credit will be awarded

Copyright © Charalambos Poullis


Academic Integrity Policy
• Feel free to discuss assignments with each other,
but coding must be done individually

• Feel free to incorporate code or tips you find on the


Web, provided this doesn’t make the assignment
trivial and you explicitly acknowledge your sources

Copyright © Charalambos Poullis


Submissions
• Electronic Assignment Submission
• https://fis.encs.concordia.ca/eas/

• Do not get confused: the dates on EAS are the late


submission deadlines

Copyright © Charalambos Poullis


Suggested Textbooks

http://szeliski.org/Book/
Copyright © Charalambos Poullis
Labs
• Start on Tuesday, 14th January 2020

• Tutorials in OpenCV
• OpenCV API

Copyright © Charalambos Poullis


Slides Acknowledgements
The slides are a combination of multiple resources and
materials generously made publicly available by

R. Szeliski, A. Efros,
S. Lazebnik,
D. Forsyth, B. Freeman, F. Durand,
J. Ponce, M. Pollefeys, L. Fei-Fei,
J. Koenderink, D. Lowe, A. Torralba,
S. Seitz, K. Grauman, R. Fergus,
J. Hays
Copyright © Charalambos Poullis
Assignments and Project
• #1: Demosaicing
• #2: Feature Detection and Matching
• Project: Image Stitching
Demosaicing
• Implement the conversion of the Bayer pixel pattern to an
RGB representation where each pixel has red, green and blue
color channels
Feature Detection and Matching
• Implement a feature detection and matching algorithm
• Feed feature matches to a structure-from-motion system

Image credit: S. Szeliski Image credit: J. Hays


Image Stitching
• Implement a feature-based alignment and image stitching
algorithm

Image credit: S. Szeliski


Copyright © Charalambos Poullis

You might also like