Scribd
Upload
94 views

YY-Deep Learning PDF

This document provides a brief introduction to deep learning, including: 1) An overview of key concepts in deep learning like artificial neural networks, backpropagation, fully connected layers, and convolutional layers. 2) A discussion of common deep learning techniques such as dropout and activation functions. 3) A brief history of major developments in deep learning research from the 1940s to recent applications of deep learning to large datasets and problems like image recognition.

Uploaded by

Abhijeet Keshri
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
94 views

YY-Deep Learning PDF

This document provides a brief introduction to deep learning, including: 1) An overview of key concepts in deep learning like artificial neural networks, backpropagation, fully connected layers, and convolutional layers. 2) A discussion of common deep learning techniques such as dropout and activation functions. 3) A brief history of major developments in deep learning research from the 1940s to recent applications of deep learning to large datasets and problems like image recognition.

Uploaded by

Abhijeet Keshri
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 46

A Brief Introduction to

Deep Learning
--Yangyan Li
How would you crack it?
How to avoid being cracked?
Seam Carving!
Labradoodle or fried chicken
Puppy or bagel
Sheepdog or mop
Chihuahua or muffin
Barn owl or apple
Parrot or guacamole
Raw chicken or Donald Trump
But, we human actually lose!
• A demo that shows we, human, lose, on the
classification task, we are proud of, we have been
trained for millions of years!
• If we want to make it hard for bots, it has to be
hard for human as well.
How would you crack it?
We human lose on Go!
We (will) lose on many specific tasks!
• Speech recognition
• Translation
• Self-driving
•…

• BUT, they are not AI yet…


• Don’t worry until it dates with your girl/boy friend…
Deep learning is so cool for so many problems…
A Brief Introduction to Deep Learning
• Artificial Neural Network

• Back-propagation

• Fully Connected Layer

• Convolutional Layer

• Overfitting
Artificial Neural Network

1. Activation function
2. Weights
3. Cost function
4. Learning algorithm

Live Demo
Neurons are functions
Neurons are functions
Back-propagation
Now, serious stuff, a bit…
Fully Connected Layers
“When in doubt, use brute force.”
--Ken Thompson
“If brute force is possible...”
--Yangyan Li
Convolutional Layers
Convolutional Layers
Convolution Filters
Feature Engineering vs. Learning
• Feature engineering is the process of using domain
knowledge of the data to create features that make
machine learning algorithms work.
• “When working on a machine learning problem,
feature engineering is manually designing what the
input x's should be.”
-- Shayne Miel
• “Coming up with features is difficult, time-
consuming, requires expert knowledge.”
--Andrew Ng
How to detect it in training process?
Dropout
Sigmod  ReLU
Sigmod  ReLU
Compute, connect, evaluate, correct, train madly…

Non-linearity, distributed representation, parallel


computation, adaptive, self-organizing…
A brief history
• McCulloch, Warren S., and Walter Pitts. "A logical calculus of the ideas immanent in nervous
activity." The bulletin of mathematical biophysics 5.4 (1943): 115-133.
• Rosenblatt, Frank. "The perceptron: a probabilistic model for information storage and
organization in the brain." Psychological review 65.6 (1958): 386.
• Rumelhart, David E., Geoffrey E. Hinton, and Ronald J. Williams. "Learning representations by
back-propagating errors." Cognitive modeling 5.3 (1988): 1.
• LeCun, Yann, et al. "Backpropagation applied to handwritten zip code recognition." Neural
computation 1.4 (1989): 541-551.
• 1993: Nvidia started…
• Hinton, Geoffrey E., Simon Osindero, and Yee-Whye Teh. "A fast learning algorithm for deep
belief nets." Neural computation 18.7 (2006): 1527-1554.
• Raina, Rajat, Anand Madhavan, and Andrew Y. Ng. "Large-scale deep unsupervised learning using
graphics processors." Proceedings of the 26th annual international conference on machine
learning. ACM, 2009.
• Deng, Jia, et al. "Imagenet: A large-scale hierarchical image database."Computer Vision and
Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009.
• 2010: “GPUS ARE ONLY UP TO 14 TIMES FASTER THAN CPUS” SAYS INTEL –Nvidia
• Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. "Deep sparse rectifier neural
networks." International Conference on Artificial Intelligence and Statistics. 2011.
• Hinton, Geoffrey E., et al. "Improving neural networks by preventing co-adaptation of feature
detectors." arXiv preprint arXiv:1207.0580 (2012).
• Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep
convolutional neural networks." Advances in neural information processing systems. 2012.
“Now this is not the end. It is not even the beginning of the
end. But it is, perhaps, the end of the beginning.”
--Winston Churchill
Is Deep Learning Taking Over the World?

• What applications are likely/unlikely to benefit


from DL? Why?
Deep learning, yay or nay?
A piece of cake,
elementary math…

It eats, a lot!

You might also like