PyTorch Tutorial

01. Overview

Lecturer : Hongpu Liu Lecture 1-1 PyTorch Tutorial @ SLAM Research Group
Goal of this tutorial

• How to implement learning system using PyTorch

• Understand the basic of neural networks / deep learning

• Requirements
• Algebra + Probability
• Python

Gao F, Huang T, Wang J, et al. Dual-Branch Deep Convolution Neural Network for Polarimetric SAR Image Classification[J]. Applied Sciences, 2017, 7(5):447.

Lecturer : Hongpu Liu Lecture 1-2 PyTorch Tutorial @ SLAM Research Group
Human Intelligence

What to eat
for dinner?

Lecturer : Hongpu Liu Lecture 1-3 PyTorch Tutorial @ SLAM Research Group
Human Intelligence

Information Infer

Lecturer : Hongpu Liu Lecture 1-4 PyTorch Tutorial @ SLAM Research Group
Human Intelligence

What to dress?

Lecturer : Hongpu Liu Lecture 1-5 PyTorch Tutorial @ SLAM Research Group
Human Intelligence

Information Infer

Lecturer : Hongpu Liu Lecture 1-6 PyTorch Tutorial @ SLAM Research Group
Human Intelligence

Image Prediction

Lecturer : Hongpu Liu Lecture 1-7 PyTorch Tutorial @ SLAM Research Group
Human Intelligence

5 0 4 1 9 2 1 3
1 4 3 5 3 6 1 7
2 8 6 9 4 0 9 1
1 2 4 3 2 7 3 8
6 9 0 5 6 0 7 4
1 8 7 9 3 9 8 5
9 3 3 0 7 4 9 8
0 9 4 1 4 4 6 0

Image Prediction

Lecturer : Hongpu Liu Lecture 1-8 PyTorch Tutorial @ SLAM Research Group
Machine learning

Information Infer

Lecturer : Hongpu Liu Lecture 1-9 PyTorch Tutorial @ SLAM Research Group
Machine learning

Image Prediction

Lecturer : Hongpu Liu Lecture 1-10 PyTorch Tutorial @ SLAM Research Group
Machine learning

5 0 4 1 9 2 1 3
1 4 3 5 3 6 1 7
2 8 6 9 4 0 9 1
1 2 4 3 2 7 3 8
6 9 0 5 6 0 7 4
1 8 7 9 3 9 8 5
9 3 3 0 7 4 9 8
0 9 4 1 4 4 6 0

Image Prediction

Lecturer : Hongpu Liu Lecture 1-11 PyTorch Tutorial @ SLAM Research Group
Machine learning

Training

Labeled Dataset

Lecturer : Hongpu Liu Lecture 1-12 PyTorch Tutorial @ SLAM Research Group
Machine learning

Lecturer : Hongpu Liu Lecture 1-13 PyTorch Tutorial @ SLAM Research Group
How to develop learning system?

Rule-based Hand-designed
Input Output
systems program

Lecturer : Hongpu Liu Lecture 1-14 PyTorch Tutorial @ SLAM Research Group
How to develop learning system?

Rule-based Hand-designed
Input Output
systems program

Classic
Hand-designed Mapping from
machine Input Output
features features
learning

Lecturer : Hongpu Liu Lecture 1-15 PyTorch Tutorial @ SLAM Research Group
How to develop learning system?

Rule-based Hand-designed
Input Output
systems program

Classic
Hand-designed Mapping from
machine Input Output
features features
learning

Mapping from
Input Features Output
features

Lecturer : Hongpu Liu Lecture 1-16 PyTorch Tutorial @ SLAM Research Group
How to develop learning system?

Rule-based Hand-designed
Input Output
systems program

Classic
Hand-designed Mapping from
machine Input Output
features features
learning

Representation
Mapping from
learning
Input Features Output
features

Lecturer : Hongpu Liu Lecture 1-17 PyTorch Tutorial @ SLAM Research Group
How to develop learning system?

Rule-based Hand-designed
Input Output
systems program

Classic
Hand-designed Mapping from
machine Input Output
features features
learning

Representation
Mapping from
learning
Input Features Output
features

Additional
Deep layers of more Mapping from
Input Simple features Output
learning abstract features
features

Lecturer : Hongpu Liu Lecture 1-18 PyTorch Tutorial @ SLAM Research Group
Rule-based system VS Representation learning

Training from data

Rule-based system

Lecturer : Hongpu Liu Lecture 1-19 PyTorch Tutorial @ SLAM Research Group
Traditional machine learning strategy

Lecturer : Hongpu Liu Lecture 1-20 PyTorch Tutorial @ SLAM Research Group
New challenge

• Limit of hand-designed feature.

• SVM can not handle big data set well.
• More and more application need to handle unstructured data.

Lecturer : Hongpu Liu Lecture 1-21 PyTorch Tutorial @ SLAM Research Group
New challenge

• Limit of hand-designed feature.

• SVM can not handle big data set well.
• More and more application need to handle unstructured data.

Deep Learning

Lecturer : Hongpu Liu Lecture 1-22 PyTorch Tutorial @ SLAM Research Group
 Brief history of neural networks
From neuroscience to mathematic & engineering

Lecturer : Hongpu Liu Lecture 1-23 PyTorch Tutorial @ SLAM Research Group
Brief history of neural networks

Cambrian
Period
543 million years, B.C.

Lecturer : Hongpu Liu Lecture 1-24 PyTorch Tutorial @ SLAM Research Group
Brief history of neural networks

Hubel D H, Wiesel T N. Receptive fields of single neurones in the cat's striate cortex[J]. Journal of Physiology, 1959, 148(3):574.

Lecturer : Hongpu Liu Lecture 1-25 PyTorch Tutorial @ SLAM Research Group
Brief history of neural networks

Single Neural Unit

Perceptron

Lecturer : Hongpu Liu Lecture 1-26 PyTorch Tutorial @ SLAM Research Group
Brief history of neural networks

Single Neural Unit Connected Neural Units

Perceptron Artificial Neural Network

Lecturer : Hongpu Liu Lecture 1-27 PyTorch Tutorial @ SLAM Research Group
Brief history of neural networks

Back Propagation

𝜕𝑒
=?
𝜕𝑎
𝜕𝑒
=?
𝜕𝑏

Lecturer : Hongpu Liu Lecture 1-28 PyTorch Tutorial @ SLAM Research Group
Brief history of neural networks

Back Propagation

𝜕𝑒
=?
𝜕𝑎
𝜕𝑒
=?
𝜕𝑏

Lecturer : Hongpu Liu Lecture 1-29 PyTorch Tutorial @ SLAM Research Group
Brief history of neural networks

Back Propagation

𝜕𝑒 𝜕𝑒 𝜕𝑐
= ∙
𝜕𝑎 𝜕𝑐 𝜕𝑎
𝜕𝑒 𝜕𝑒 𝜕𝑐 𝜕𝑒 𝜕𝑑
= ∙ + ∙
𝜕𝑏 𝜕𝑐 𝜕𝑏 𝜕𝑑 𝜕𝑏

Lecturer : Hongpu Liu Lecture 1-30 PyTorch Tutorial @ SLAM Research Group
Brief history of neural networks

LeNet-5 LeCun 1998

Y. LeCun, L. Bottou, Y. Bengio and P. Haffner: Gradient-Based Learning Applied to Document Recognition, Proceedings of the IEEE, 86(11):2278-2324, November 1998,

Lecturer : Hongpu Liu Lecture 1-31 PyTorch Tutorial @ SLAM Research Group
Brief history of neural networks

[1998] [2012] [2014] [2015]

LeNet-5 AlexNet GoogLeNet & VGG ResNet

Lecturer : Hongpu Liu Lecture 1-32 PyTorch Tutorial @ SLAM Research Group
Brief history of neural networks

Algorithm Data Computation

Lecturer : Hongpu Liu Lecture 1-33 PyTorch Tutorial @ SLAM Research Group
Good news

• Deep learning is not too difficult

• Basic algebra + probability + python
• Less than one year study
• There are lots of deep learning framework
• Starting from scratch do not be required
• Enabled efficient and convenient use of GPU
• Lots of components of neural networks provided by framework
• Popular deep learning frameworks
• Theano (University of Montreal) / TensorFlow (Google)
• Caffe (UC Berkeley) / Caffe 2 (Facebook)
• Torch (NYU & Facebook) / PyTorch (Facebook)

Lecturer : Hongpu Liu Lecture 1-34 PyTorch Tutorial @ SLAM Research Group
What is PyTorch

• PyTorch is a python package that provides two high-level

features:
• Tensor computation (like numpy) with strong GPU acceleration
• Deep Neural Networks built on a tape-based autodiff system

Lecturer : Hongpu Liu Lecture 1-35 PyTorch Tutorial @ SLAM Research Group
Why PyTorch

• Dynamical graph
• More flexible
• Easy to debug
• Intuitive and cleaner code
• More neural networkic
• Write code as network
works
• AutoGrad for forward /
backward

Lecturer : Hongpu Liu Lecture 1-36 PyTorch Tutorial @ SLAM Research Group
Install PyTorch

https://pytorch.org

Lecturer : Hongpu Liu Lecture 1-37 PyTorch Tutorial @ SLAM Research Group
After install PyTorch on your computer

PS C:\Users\liuii> python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> print(torch.__version__)
0.4.0
>>> # Perfect!

Lecturer : Hongpu Liu Lecture 1-38 PyTorch Tutorial @ SLAM Research Group
 PyTorch Tutorial
01. Overview

Lecturer : Hongpu Liu Lecture 1-39 PyTorch Tutorial @ SLAM Research Group