INTRODUCTION TO

DEEP LEARNING
CS617: Basics of Deep Learning
Instructor: Aparajita Ojha
Slide1
 About the Course
• 4 credit Course
• Grading policy:
– Relative grading
– Quiz-10%, Project 20%, Midesem-30%, Endsem-40%
– Academic honesty: Grade F if used any kind of unfair means in
any of the quiz\ test\ project.
• Course details: available on my course page
web.iiitdmj.ac.in/~aojha

2 CS617 Basics of Deep Learning 1/18/2019

 Course Overview
• Introduction to supervised learning, neural networks and
deep learning, Logistic regression, gradient descent,
vectorization, logistic regression cost function, python
programming of logistic regression gradient descent and cost
function.
• Activation functions, Forward and Backward propagation,
parameters and hyperparameters, Improving deep neural
networks, hyperparameters tuning.
• Regularization and optimization, Bias and variance, Dropout
regularization, Normalizing input, Weight initialization,
Gradient checking.
• Vanishing and Exploding Gradient Problems, Autoencoders
• Optimization, Mini batch gradient descent, Exponentially
weighted average, Adam optimization.

3 CS617 Basics of Deep Learning 1/18/2019

 Course Overview…
• Brief introduction to Tensor flow, Keras and
implementation of algorithms.
• Convolution neural network (CNN), ResNets, Applications
of CNN to computer vision and object detection.
• Recurrent Neural Networks (RNNs), Forward Propagation
and Backward propagation in RNNs, Implementation of
an RNN, GRUs (Gated Recurrent Units) and LSTMs (Lon
Short Term Memory Networks)

4 CS617 Basics of Deep Learning 1/18/2019

 Can AI surpass
human Motivation
intelligence ?

5 CS617 Basics of Deep Learning 1/18/2019

 Motivation
Can software
robots be more
intelligent in
their tasks ?

6 CS617 Basics of Deep Learning 1/18/2019

 Deep Learning Impact
• Nature , January 2016 and October 2017
• “DeepMind’s program AlphaGo beat Fan Hui, the European Go
champion, five times out of five in tournament conditions...”

• The Difference in Approaches

– The IBM chess computer Deep Blue, Explicitly programmed to win

against the grandmaster Garry Kasparov in 1997
– “But AlphaGo was not preprogrammed to play Go.
– It learned using a general-purpose algorithm that allowed it to
interpret the game’s patterns.
• AlphaGo program applied deep learning in neural networks
(convolutional NN) —in which connections between layers of
neurons are strengthened through examples and experience.”
7 CS617 Basics of Deep Learning 1/18/2019
 Deep Learning Impact
• Google Assistant getting more powerful

• Machine translation improving

• Image search giving better results

• Speech recognition improving

8 CS617 Basics of Deep Learning 1/18/2019

 Deep Learning Impact

Reference: Kaiming He, Xiangyu Zhang, Shaoqing Ren, & Jian Sun, 2015 ; Fei Fei Li, Lecture Slides, Introduction to CNN
9 CS617 Basics of Deep Learning 1/18/2019
 Deep Learning Today

source:developer.nvidia.com/deep--‐learning--‐courses
10 CS617 Basics of Deep Learning 1/18/2019
 Introduction
• What is deep learning ?
• “ A machine learning technique that allows computers to
improve with experience and data”.
• “Deep learning is a particular kind of machine learning
that achieves great power and flexibility by learning to
represent the world as a nested hierarchy of concepts,
with each concept defined in relation to simpler concepts,
and more abstract representations computed in terms of
less abstract ones”.
– From the Book on Deep Learning by Ian Goodfellow, Yoshua
Bengio, Aaron Courville, MIT Press, 2017.

11 CS617 Basics of Deep Learning 1/18/2019

 Machine Learning (ML)
• Machine learning is a field of computer science that gives
computers the ability to automatically learn without
being explicitly programmed.
• Learning from experience on data to make predictions.

Machine Learning
Data
algorithm

Training
Prediction

Data Trained model Prediction

12 CS617 Basics of Deep Learning 1/18/2019

 Deep Learning…

Figure Source: Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville, MIT Press, 2017.
13 CS617 Basics of Deep Learning 1/18/2019
 Machine Learning Algorithm
• A machine learning algorithm is an algorithm that is
able to learn and extract patterns from data.
• Learning –
– A computer program is said to learn from
• experience E
• with respect to some class of tasks T
• and performance measure P
– if its performance at tasks in T , as measured by P ,
improves with experience E.” (Mitchell , 1997)

14 CS617 Basics of Deep Learning 1/18/2019

 Learning Algorithms…
• Tasks
– Classification, Regression, Transcription , Machine Translation etc.
• Performance measures
– Depends on the type of problem: Examples include –
• accuracy, error rate etc.
– Performance is measured on a dataset called test dataset, that is
different from the dataset used to train the algorithms.
– Often difficult to choose a performance measure that corresponds
well to the desired behavior of the system.
• Experience
– Algorithms are termed as supervised learning or unsupervised
learning algorithms based on the experience they are allowed to have
on datasets.

15 CS617 Basics of Deep Learning 1/18/2019

 Learning Algorithms…
• Three broad categories of learning

Semi-
supervised
learning

Figure source: https://developer.ibm.com/articles/cc-models-machine-learning/

16 CS617 Basics of Deep Learning 1/18/2019
 Learning Algorithms…
• Supervised learning
– observing several examples of a random vector x and an associated
value or vector y, and learning to predict y from x, usually by
estimating the probability of occurrence of y under the condition that
x has occurred.
– the target y being provided by an instructor who shows the machine
learning system what to do.
– Two types – generative and discriminative.

• Unsupervised Learning
– observing several examples of a random vector x, and attempting to
implicitly or explicitly learn the probability distribution p(x), or some
interesting properties of that distribution
– there is no instructor and the algorithm must learn to make sense of
the data without this guide.

17 CS617 Basics of Deep Learning 1/18/2019

 Supervised Learning

Figure source: Lecture slides of Yi-Fan Chang

18 CS617 Basics of Deep Learning 1/18/2019
 Unsupervised Learning

Figure source: Lecture slides of Yi-Fan Chang

19 CS617 Basics of Deep Learning 1/18/2019
 Learning Algorithms…
• Semi Supervised learning
• a class of supervised learning algorithms that also make
use of unlabeled data for training
– In addition to unlabeled data, algorithms are provided with
some supervision information –not for all examples.

• Reinforcement learning
– Such algorithms interact with an environment.
– There is a feedback loop between the learning system and its
experiences.

20 CS617 Basics of Deep Learning 1/18/2019

 Limitations of Traditional ML
Systems
Curse of dimensionality

Many machine
learning problems
become
exceedingly difficult
when the number
of dimensions in
the data is high.

21 CS617 Basics of Deep Learning 1/18/2019

 Curse of Dimensionality
• Hughes phenomenon (or peaking phenomenon)
• With a fixed number of training samples, the predictive
power of a classifier or regressor first increases as the
number of dimensions/features used is increased but
then decreases.
• Examples
– Speech recognition
– Object detection in images

22 CS617 Basics of Deep Learning 1/18/2019

 Limitations of Traditional ML
Systems
• Traditional machine learning uses manually designed features
that are
– Often over-specified and incomplete.
– Mostly application-dependent, cannot be generalized easily to
other applications.
– Time consuming in design and validation.
• Developing hand-crafted features is a costly and complex job.

hand-crafted
Trainable
feature output
extractor system

23 CS617 Basics of Deep Learning 1/18/2019

 Deep Learning
• Deep learning uses multiple levels of feature learning to
learn rich hierarchical representations (features)
automatically.
• Provides a flexible learnable framework for representing
audio, visual or linguistic information.
• Trained algorithms work for a broad range of problems.
Low-level Mid-level High-level Trainable
features features features system output

Figure from: Zeiler and Fergus, 2013: Feature visualization of convolution NN on ImageNet database.
24 CS617 Basics of Deep Learning 1/18/2019
 Machine Learning vs Deep Learning

Figure source: Jinjiang Wang ‘s page on researchgate.net

25 CS617 Basics of Deep Learning 1/18/2019

 Deep Learning Success Mantra

• Increased availability of data

• Significant improvement in computing power

• Increased model size

• New way of learning representations ( auto-feature

extraction)

• Use of Deep Neural Network Architectures that generalize

well and are flexible enough to address a broad range of tasks.

26 CS617 Basics of Deep Learning 1/18/2019

 Deep Learning: Basic Building Blocks

• Artificial Neural Networks have been the basic

building blocks for deep learning.

• Artificial neural networks (ANNs) are based on the

concept of a perceptron.

• Let us have a glimpse of the history of Deep

Learning.

27 CS617 Basics of Deep Learning 1/18/2019

 ANN: A Brief History
• Initial concept building
1890: William James - defined a neuronal process of
learning

• Early technology development Initiatives

1943: McCulloch and Pitts - earliest mathematical
models
1954: Donald Hebb and IBM research group - earliest
simulations
1958: Frank Rosenblatt - The Perceptron

28 CS617 Basics of Deep Learning 1/18/2019

 Neurons in the Brain
• As the very basic level the brain is composed of neurons
– A neuron receives input from other neurons (generally thousands)
from its synapses.
– Inputs are combined ( approximately added )
– When the input exceeds a threshold the neuron sends an electrical
spike that travels down the axon to the next neuron(s)

29 CS617 Basics of Deep Learning 1/18/2019

 Artificial Neural Network (ANN)
• An information processing paradigm inspired by human
brain’s information processing mechanism.

• Composed of a large number of highly interconnected

processing elements called neurons working in unison to
solve specific problems.
– ANNs learn by examples like human being.

• Until recently, ANNs were normally configured through a

learning algorithm for specific applications such as
– pattern recognition, data classification

• Learning in human brain involves adjustments to the

synaptic connections that exist between neurons. ANNs
are modeled in the same way.
30 CS617 Basics of Deep Learning 1/18/2019
 Let us see the classical
Rosenblatt’s Perceptron Model

31 CS617 Basics of Deep Learning 1/18/2019

 Rosenblatt’s Perceptron Model
• Rosenblatt’s Perceptron machine for binary classification
• Multiply a weight for each input.
• Add bias =+1
• If result is above a pre-set threshold return 1, else 0.
weights
x1 Use a
w1 threshold
Net
x2 w2 input
Input
෍ 𝝈 Output
x3 Activation
w3 Transfer
function function
x4
w4

32 CS617 Basics of Deep Learning 1/18/2019

 Rosenblatt Perceptron Algorithm
• Input: A sequence of labelled examples
– 𝒙𝟏 , 𝑦1 , 𝒙𝟐 , 𝑦2 , , … , (𝒙𝒏 , 𝑦𝑛 , ), where each 𝒙𝒊 ∈ 𝑹𝒅 and 𝑦𝑖 ∈
−1, 1 ,
• Find a weight vector 𝒘 and intercept 𝑏 such that
𝑠𝑖𝑔𝑛 𝒘𝒙𝒊 + 𝑏 = 𝑦𝑖 for all 𝑖
• Perceptron Algorithm
– Initialize 𝒘 = 0
– if 𝑠𝑖𝑔𝑛 𝒘𝒙 + 𝑏 ≠ 𝑦 (error), then 𝑥 = [𝒙𝟏 , 𝒙𝟐 … , 𝒙𝒏 ]
– update 𝒘 𝑦 = [𝑦1 , 𝑦2 … , 𝑦𝑛 ] 𝑇

• 𝒘 = 𝒘 + 𝒙 𝑦 ( is learning rate )

33 CS617 Basics of Deep Learning 1/18/2019

 Current Neuron Model
Input
weight

Activation function
𝑥1
𝑤1

𝑥2 𝑤2 ෍
f(x) output
𝑤3

𝑥3 Summing up the input

34 CS617 Basics of Deep Learning 1/18/2019
 Current Neuron Model…
1
𝜎 𝑧 =
1 + 𝑒 −𝑧
An Example
-0.06 Activation function
0.7

-2.5 -1.6
෍ (x)
0.002

Weighted sum
1.4 x = -0.06×2.7 + 2.5×8.6 + 1.4×0.002 = 4.0048
𝜎 𝑥 = 0.982
35 CS617 Basics of Deep Learning 1/18/2019
 Modelling ‘AND’
• Let us see how does a neuron model reconstructs ‘AND’
operation.
x1 x2 Output 𝑤1 𝑥1 + 𝑤2 𝑥2 + 𝑏 = 𝐿𝑎𝑏𝑒𝑙
(Label )
0 0 0 𝑤1 . 0 + 𝑤2 . 0 + 𝑏 = 0
0 1 0 𝑤1 . 0 + 𝑤2 . 1 + 𝑏 = 0
1 0 0 𝑤1 . 1 + 𝑤2 . 0 + 𝑏 = 0
1 1 1 𝑤1 . 1 + 𝑤2 . 1 + 𝑏 = 1
Can we suitably adjust weight and
𝐴𝑐𝑡𝑖𝑣𝑎𝑡𝑖𝑜𝑛 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 bias parameters so that all four
1 𝑧≥𝜃 equations are satisfied ?
𝑓 𝑧 = ൜
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Yes, for example if 𝜃 = 0.5, choose 𝑤1 = 0.3 = 𝑤2 , 𝑏 = 0.
36 CS617 Basics of Deep Learning 1/18/2019
 General Perceptron Algorithm …
• Dealing with XOR: (Minsky, Papert, 1969)

Figure from: Prof. Efstratios Gavves’s lecture slides on UVA Deep Learning Course
37 CS617 Basics of Deep Learning 1/18/2019
 Limitations and Slow Down
• Single layer of perceptron are not able to solve
complex tasks.
– Linear decision boundaries.

38 CS617 Basics of Deep Learning 1/18/2019

 Multilayer Perceptron
• Universal Approximation Theorem (Cybenko, 1989)
– Any continuous function over a compact domain can be
accurately approximated using a neural network with one
hidden layer only.
• If layers increase, accuracy of approximation increases.
• Multi-layer perceptron models were introduced to deal
with nonlinearity and for better approximation of
complex functions.

39 CS617 Basics of Deep Learning 1/18/2019

 Feed Forward NNs
• Feed forward neural networks are the Computes its
basic building blocks of NNs and deep own activation
learning. value

f1 f2 f3 f4
• The model is represented with a directed
acyclic graph describing how the functions
are composed together.

• The function
𝑓 𝑥 = 𝑓 4 (𝑓 3 (𝑓 2 (𝑓 1 (𝑥))))
tends to approximate 𝑓 ∗ .

• For each input x, the NN should

approximate the corresponding y value.

• NN is trained to minimize the error of Width

approximation.
40 CS617 Basics of Deep Learning 1/18/2019
 Feed Forward NNs
• Essentially a feedforward network is a function
approximation machine that is designed to achieve
statistical generalization.
– Occasionally drawing some insights from what we know
about the brain, rather than as a model of brain functions.
- Ian Goodfellow’s book

41 CS617 Basics of Deep Learning 1/18/2019

 Feed Forward NNs

Prediction of the corresponding y

Output Layer (Decision Layer) . . .

. . .
Hidden Layer(s)
. . .

Input Layer . . .

x1 x2 xm
42 CS617 Basics of Deep Learning
Input sample x 1/18/2019
 Pattern Classification
• Function: x  y
output pattern y
y1 y2 yn
• The NN’s output is used to
recognize different input patterns. . . .
• Different output patterns
correspond to particular classes of . . .
input patterns.
• Networks with hidden layers can . . .
be used for solving more complex
problems than just a linear pattern . . .
classification.
x1 x2 xm
input pattern x
43 CS617 Basics of Deep Learning 1/18/2019
 Generalization
• By proper training, a neural network
may produce reasonable answers for
input patterns not seen during y2pattern 𝑦ොyn
yOutput
1
training (generalization).
. . .

• Generalization is particularly useful . . .

for the analysis of a “noisy” data.
. . .

. . .

x1 x2 xm
44 CS617 Basics of Deep Learning
Input pattern x 1/18/2019
 An Example

ANN with 1 Hidden Layer

45 CS617 Basics of Deep Learning 1/18/2019

 ANN with 1 Hidden Layer

Train the NN
Initialise with random weights

Features class
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0
etc …

46 CS617 Basics of Deep Learning 1/18/2019

 ANN with 1 Hidden Layer…

Train the NN
Initialise with random weights
Present a training pattern and feed it to the NN.
Get an output and compare with actual value.
Training Data
Features class
1.4 2.7 1.9 0
3.8 3.4 3.2 0
6.4 2.8 1.7 1
4.1 0.1 0.2 0
etc …

47 CS617 Basics of Deep Learning 1/18/2019

 ANN with 1 Hidden Layer…

Train the NN
Initialise with random weights
Present a training pattern and feed it to the NN.
Get an output and compare with actual value.
Training Data
Features class 1.4
1.4 2.7 1.9 0
3.8 3.4 3.2 0 2.7 0.8
6.4 2.8 1.7 1
4.1 0.1 0.2 0 1.9
etc …

48 CS617 Basics of Deep Learning 1/18/2019

 ANN with 1 Hidden Layer…

Train the NN
Initialise with random weights
Present a training pattern and feed it to the NN.
Get an output and compare with actual value.
Training Data
Features class 1.4
1.4 2.7 1.9 0
3.8 3.4 3.2 0 2.7 0.8
6.4 2.8 1.7 1
4.1 0.1 0.2 0 1.9
etc …
Expected output
49 CS617 Basics of Deep Learning 1/18/2019
 ANN with 1 Hidden Layer…

Train the NN
Initialise with random weights
Present a training pattern and feed it to the NN.
Get an output and compare with actual value.
Training Data
Features class 1.4
1.4 2.7 1.9 0
3.8 3.4 3.2 0 2.7 0.8
6.4 2.8 1.7 1
4.1 0.1 0.2 0 1.9
etc …
Expected output Error = 0.8
50 CS617 Basics of Deep Learning 1/18/2019
 ANN with 1 Hidden Layer…
Train the NN
Initialise with random weights
Present a training pattern and feed
Present a training pattern and feed it to the NN.
Get an output and compare with actual value.
Adjust and update the weights.
Training Data
Features class 1.4
1.4 2.7 1.9 0
3.8 3.4 3.2 0 2.7 0.8
6.4 2.8 1.7 1
4.1 0.1 0.2 0 1.9
etc …
Expected output Error = 0.8
51 CS617 Basics of Deep Learning 1/18/2019
 ANN with 1 Hidden Layer…
Train the NN
Initialise with random weights
Present a training pattern and feed it to the NN.
Get an output and compare with actual value.
Adjust and update the weights.
Present a training pattern and feed it to the NN
Training Data
Features class 6.4
1.4 2.7 1.9 0
3.8 3.4 3.2 0 2.8 0.9
6.4 2.8 1.7 1
4.1 0.1 0.2 0 1.7
etc …
Training pattern Error = - 0.1
52 CS617 Basics of Deep Learning 1/18/2019
 ANN with 1 Hidden Layer…
Initialise with random weights
Present a training pattern and feed it to the NN.
Get an output and compare with actual value.
Adjust and update the weights.
Present a training pattern and feed it to the NN.
Repeat the process to reduce the error.
Training Data
Features class 4.1
1.4 2.7 1.9 0
3.8 3.4 3.2 0 0.1
6.4 2.8 1.7 1
4.1 0.1 0.2 0 0.2
etc …
Next training pattern
53 CS617 Basics of Deep Learning 1/18/2019
 Mathematical Model: ANN…
• Weight adjustment process is repeated thousands and
thousands of times.
• Each time a random training example is taken, weights
are slightly adjusted to tune the system for reducing
error of approximation.
• It may not be an efficient adjustment on many other
cases.
• But eventually the process of weight adjustments leads
to a good enough model for producing an effective
classifier.
• It works well in many real applications.
54 CS617 Basics of Deep Learning 1/18/2019
 Further Advancements
• Back propagation
• Sophisticated algorithms
– Recurrent Long-Short Term Memory Networks
(Hochreiter and Schmidhuber in 1997 )
– Optical Character recognition using Convolutional
Neural Networks (Yann LeCun et al. late 1990s)

55 CS617 Basics of Deep Learning 1/18/2019

 Back Propagation

Figure source: G.E. Hinton’ lecture slide

56 CS617 Basics of Deep Learning 1/18/2019
 Network Training
Sample labeled Forward it Back- Update the
data through the
propagate the network
network, get
(batch) errors weights
predictions

Optimize (min. or max.) objective/cost function 𝑱(𝑾, 𝒃)

Generate error signal that measures difference between
predictions and target values.

Use error signal to change the weights and get more

accurate predictions.
Subtracting a fraction of the gradient moves you
towards the (local) minimum of the cost function.
https://medium.com/@ramrajchandradevan/the-evolution-of-gradient-descend-optimization-algorithm-4106a6702d39
57 CS617 Basics of Deep Learning 1/18/2019
 Second Slowdown: 1990-2006
• Limitations with processing power
• Experimentally, training multi-layer perceptrons was
not that useful
– Accuracy didn’t improve with more layers.
• Not enough data for training MLPs.
– Result: Overfitting, large generalization errors
• Vanishing gradient
– While learning with NN, we need to multiply several gradient
values, and if the values are less than 1, for deep NNs, they tend
to rapidly vanish, leaving no scope of improvements in training
through backpropagation.
58 CS617 Basics of Deep Learning 1/18/2019
 Mental Break!
• A Funny Artificial Intelligence Failure!
• Robot passport checker rejects an Asian man’s application because
“eyes are closed.”

To err isn't just human -- machines make mistakes, too!

Image Source: https://www.entrepreneur.com
59 CS617 Basics of Deep Learning 1/18/2019
 Deep Belief Network
• A simple way to train deep networks to perform better and
fast.
• Hinton et al. , 2006.
– A deep belief network (DBN) is a generative graphical model.
– A class of deep neural networks, composed of multiple layers of latent
variables (hidden units).
– Connections between layers but not between units within each layer.
– When trained on a set of examples without supervision, a DBN can
learn to probabilistically reconstruct its inputs.
– The layers then act as feature detectors.
– After the learning, a DBN is further trained with supervision to
perform classification.
• This led to the development of deep neural networks as
powerful tools for solving many challenging problems like
– Translation, object detection in images, speech recognition.
60 CS617 Basics of Deep Learning 1/18/2019
 DBN

From: Prof. David Wolfe Corne’s lecture slides

61 CS617 Basics of Deep Learning 1/18/2019
 DBN: New Training Method

Train this layer first

From: Prof. David Wolfe Corne’s lecture slides

62 CS617 Basics of Deep Learning 1/18/2019
 DBN: New Training Method…

then this layer

From: Prof. David Wolfe Corne’s lecture slides

63 CS617 Basics of Deep Learning 1/18/2019
 DBN: New Training Method…

then this layer

From: Prof. David Wolfe Corne’s lecture slides

64 CS617 Basics of Deep Learning 1/18/2019
 DBN: New Training Method…

then this layer

From: Prof. David Wolfe Corne’s lecture slides

65 CS617 Basics of Deep Learning 1/18/2019
 DBN: New Training Method…

finally this layer

From: Prof. David Wolfe Corne’s lecture slides

66 CS617 Basics of Deep Learning 1/18/2019
 DBN: New Training Method…

EACH of the (non-output) layers is trained to

be an auto-encoder
Each layer is made to learn good features
that describe what is received from the
previous layer

67 CS617 Basics of Deep Learning From: Prof. David Wolfe Corne’s lecture1/18/2019
slides
 Summary
• Deep Learning Technologies have brought significant
improvement in performances of machines in various
complex tasks - machine translations, object detection, data
hiding etc.
• Deep learning frameworks have evolved from ANN based
machine learning methods.
• Deep learning differs from traditional ML methods in the way
network are trained to automatically detect features in a
hierarchical way- No need to manually select features!
• Various Deep learning architectures have been proposed over
the years.

68 CS617 Basics of Deep Learning 1/18/2019

 References
• Slides are based on, and some figures \captions taken from
the following resources.
– Ian Goodfellow, Deep Learning, MIT Press, 2016.
– Andrew Ng Lectures on Introduction to Neural Network and Deep
Learning ( Coursera.org)
– Kaiming He, Xiangyu Zhang, Shaoqing Ren, & Jian Sun, Deep Residual
Networks for Image Recognition, 2015 ; arxiv.org
– Fei Fei Li, Lecture Slides, Introduction to CNN, Stanford University.
– Efstratios Gavves, Lecture slides on UVA Deep Learning Course
– Yingyu Liang, Lecture slides on Basics of Deep Learning Course.
– Tai-Wen Yue, Lecture slides on Artificial Neural Networks.
– G.E. Hinton’ lecture slides on CNN.

69 CS617 Basics of Deep Learning 1/18/2019