CS231n:

Deep Learning for Computer Vision

Lecture 1 - Overview

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 1 March 29, 2022
 Today’s agenda

● A brief history of computer vision

● CS231n overview

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 2 March 29, 2022
 Today’s agenda

● A brief history of computer vision

● CS231n overview

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 3 March 29, 2022
 Image Classification: A core task in Computer Vision

cat

This image by Nikita is

licensed under CC-BY 2.0

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 4 March 29, 2022
 There are many visual recognition problems that
are related to image classification, such as
object detection, image captioning, image
segmentation, visual question answering, visual
instruction navigation, video understanding, etc.

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 5 March 29, 2022
 Deep Learning for
Computer Vision

Hierarchical computing systems with many “layers”, that are very loosely
inspired by Neuroscience

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 6 March 29, 2022
 Neural Networks

x W1 h W2 s cat

3072 100 10

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 7 March 29, 2022
 Convolutional Neural Networks
for Visual Recognition

A class of Neural Networks that have become an

important tool for visual recognition

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 8 March 29, 2022
 Beyond Convolutional Neural Networks

Recurrent neural network Attention mechanism / Transformers

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 9 March 29, 2022
 Beyond Image Classification
Semantic Object Instance
Classification
Segmentation Detection Segmentation

CAT GRASS, CAT, DOG, DOG, CAT DOG, DOG, CAT

TREE, SKY

No spatial extent No objects, just pixels Multiple Object This image is CC0 public domain

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 10 March 29, 2022
 Beyond 2D Images

Simonyan and Zisserman, “Two-stream convolutional networks for action

recognition in videos”, NeurIPS 2014

Gkioxari et al., “Mesh R-CNN”, ICCV 2019

Choy et al., 3D-R2N2: Recurrent Reconstruction Neural Network (2016)

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 11 March 29, 2022
 Beyond Vision

Mandlekar and Xu et al., Learning to Generalize Across

Gao et al., ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer (2022) Long-Horizon Tasks from Human Demonstrations (2020)

Wang et al., 6-PACK: Category-level 6D Pose Tracker with

Xu et al., PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation (2018) Anchor-Based Keypoints (2020)

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 12 March 29, 2022
 2018 Turing Award for deep learning
most prestigious technical award, is given for major contributions of lasting importance to computing.

Jeffrey Hinton Yoshua Bengio Yann LeCun

This image is CC0 public domain This image is CC0 public domain This image is CC0 public domain

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 13 March 29, 2022
 IEEE PAMI Longuet-Higgins Prize
Award recognizes ONE Computer Vision paper from ten years ago with significant impact on computer
vision research.

At CVPR 2019, it was awarded to the 2009 original ImageNet paper

That’s Fei-Fei

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 14 March 29, 2022
 2020

2021

>8k submissions, 2,067 accepted papers

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 15 March 29, 2022
 Logistics

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 16 March 29, 2022
Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 17 March 29, 2022
 Lectures

- Tuesdays and Thursdays between 1:30pm to 3:00pm at NVIDIA Auditorium

- Slides will be posted on the course website shortly before each lecture

- All lectures will be recorded and uploaded to Canvas after the lecture under the
“Panopto Course Videos” Tab.

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 18 March 29, 2022
 Course website

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 19 March 29, 2022
 Friday Discussion Sections
6 Discussion sections Fridays 1:30pm - 2:30pm over Zoom
04/01 Python / Numpy Review Session

04/08 Backprop Review Session

04/15 Final Project Overview and Guidelines

04/22 PyTorch / TensorFlow Review Session

04/29 Detection software & RNNs

05/06 Midterm Review Session

Hands-on tutorials, with more practical details than the main lecture

Check canvas for the Zoom link of the discussion sessions!

This Friday: Python / numpy / Colab (Presenter: Manasi Sharma)

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 20 March 29, 2022
 Ed

For questions about midterm, projects, logistics, etc, use Ed!

SCPD students: Use your @stanford.edu address to register for Ed; contact
[email protected] for help.

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 21 March 29, 2022
 Office Hours
We'll be using Zoom to hold office hours and QueueStatus to setup queues

- please see Canvas or Ed for the QueueStatus link

- TAs will admit students to their Zoom meeting rooms for 1-1 conversations
when it’s your turn using QueueStatus.
- Office hours is listed on the course webpage!

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 22 March 29, 2022
 Optional textbook resources
- Deep Learning
- by Goodfellow, Bengio, and Courville
- Here is a free version
- Mathematics of deep learning
- Chapters 5, 6 7 are useful to understand vector calculus and continuous optimization
- Free online version
- Dive into deep learning
- An interactive deep learning book with code, math, and discussions, based on the NumPy
interface.
- Free online version

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 23 March 29, 2022
 Assignments
All assignments will be completed using Google Colab

Assignment 1: Will be out Friday, due 4/15 by 11:59pm

- K-Nearest Neighbor
- Linear classifiers: SVM, Softmax
- Two-layer neural network
- Image features

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 24 March 29, 2022
 Grading
All assignments, coding and written portions, will be submitted via Gradescope.

An auto-grading system:

- A consistent grading scheme

- Public tests:
- Students see results of public tests immediately
- Private tests
- Generalizations of the public tests to thoroughly test your implementation

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 25 March 29, 2022
 Grading
3 Assignments: 10% + 20% + 15% = 45%
In-Class Midterm Exam: 20%
Course Project: 35%
- Project Proposal: 1%
- Milestone: 2%
- Final Project Report: 29%
- Poster & Poster Session: 3%

Participation Extra Credit: up to 3%

Late policy
- 4 free late days – use up to 2 late days per assignment
- Afterwards, 25% off per day late
- No late days for project report

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 26 March 29, 2022
 AWS

We will have AWS Cloud credits available for projects

- Not for HWs (only for final projects)

We will be distributing coupons to all enrolled students who need it

We will have a tutorial for walking through the AWS setup

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 27 March 29, 2022
 Overview on communication
Course Website: http://cs231n.stanford.edu/
- Syllabus, lecture slides, links to assignment downloads, etc
Ed:
- Use this for most communication with course staff
- Ask questions about homework, grading, logistics, etc
- Use private questions only if your post will violate honor code if you release publicly.

Gradescope:
- For turning in homework and receiving grades
Canvas:
- For watching recorded lectures
- For watching recorded discussion sessions

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 28 March 29, 2022
 Prerequisites
Proficiency in Python
- All class assignments will be in Python (and use numpy)
- Later in the class, you will be using Pytorch and TensorFlow
- A Python tutorial available on course website
College Calculus, Linear Algebra
No longer need CS229 (Machine Learning)

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 29 March 29, 2022
 Collaboration policy
We follow the Stanford Honor Code and the CS Department Honor Code – read
them!
● Rule 1: Don’t look at solutions or code that are not your own; everything you
submit should be your own work
● Rule 2: Don’t share your solution code with others; however discussing ideas
or general strategies is fine and encouraged
● Rule 3: Indicate in your submissions anyone you worked with
Turning in something late / incomplete is better than violating the honor code

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 30 March 29, 2022
 Learning objectives
Formalize computer vision applications into tasks
- Formalize inputs and outputs for vision-related problems
- Understand what data and computational requirements you need to train a model
Develop and train vision models
- Learn to code, debug, and train convolutional neural networks.
- Learn how to use software frameworks like PyTorch and TensorFlow
Gain an understanding of where the field is and where it is headed
- What new research has come out in the last 0-5 years?
- What are open research challenges?
- What ethical and societal considerations should we consider before deployment?

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 31 March 29, 2022
 Why should you take this class?
Become a vision researcher (an incomplete list of conferences)
- Get involved with vision research at Stanford: apply using this form.
- CVPR 2022 conference
- ICCV 2021 conference
Become a vision engineer in industry (an incomplete list of industry teams)
- Perception team at Google AI, Vision at Google Cloud
- Vision at Meta AI
- Vision at Amazon AWS
- Nvidia, Tesla, Apple, Salesforce, ……
General interest

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 32 March 29, 2022
 CS231n: Deep Learning for Computer Vision

● Deep Learning Basics (Lecture 2 – 4)

● Perceiving and Understanding the Visual World (Lecture 5 – 12)

● Reconstructing and Interacting with the Visual World (Lecture 13 – 16)

● Human-Centered Artificial Intelligence (Lecture 17 – 18)

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 33 March 29, 2022
 Syllabus

Deep Learning Basics Convolutional Neural Networks Computer Vision Applications

Data-driven learning Convolutions RNNs / Attention / Transformers

Linear classification & kNN PyTorch / TensorFlow Image captioning
Loss functions Activation functions Object detection and segmentation
Optimization Batch normalization Style transfer
Backpropagation Transfer learning Video understanding
Multi-layer perceptrons Data augmentation Generative models
Neural Networks Momentum / RMSProp / Adam Self-supervised learning
Architecture design 3D vision
Human-centered AI
Fairness & ethics

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 34 March 29, 2022
 Next time: Image classification with Linear Classifiers
k- nearest neighbor Linear classification

Plot created using Wolfram Cloud

Fei-Fei Li, Jiajun Wu, Ruohan Gao Lecture 1 - 35 March 29, 2022