DASC7606-1A

Deep Learning
Motivation and Introduction
Dr. Bethany Chan
Professor Francis Chin
2023

1
 Outline
• The story behind Deep Learning
• Applications
• Drivers of the recent breakthroughs

2
 The story behind Deep Learning

Source: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/
3
 Alan Turing

• Biographical film Imitation Game (2014)

WWII codebreaking
using ENIGMA
• Imitation Game
= Turing Test

®“Computing Machinery and Intelligence” (Oct 1950)

4
 Turing Test: an imitation game

OR

ANSWER

A. M. Turing. Computing machinery and intelligence. Mind, 59(236):433–460, 1950.

5
 Turing Test: an imitation game
To pass the Turing
Test, the machine
would have to imitate
a human (and fool the
interrogator to believe
he is interacting with
a human).
If a machine passed
this test, it would be
considered intelligent.

A. M. Turing. Computing machinery and intelligence. Mind, 59(236):433–460, 1950.

6
ELIZA (MIT Prof. Joseph Weizenbaum, mid 1960s)

the first psychotherapy chatbot

Standard opening Same response to all answers

Spot the keyword
“something”

Type back

Spot “depressed”

Spot “unhappy”
7
 Forbes June 2018
ChatCBC June 2014 Amazon Echo, NLP

May 2018
Speech + Booking appointment
 The story behind Deep Learning

Human
Intelligence
Exhibited by
Machines

Source: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/
9
“Instead of trying to produce a programme to
simulate the adult mind, why not rather try to
produce one which simulates the child's?
If this were then subjected to an appropriate course
of education one would obtain the adult brain.”
- Alan Turing
® Learning as the basis of intelligence
® Start from a child’s brain
Tabula rasa - idea that people are born without
built-in mental content and therefore all knowledge
is learnt
® Get more intelligent through time
® interative improvement
10
How did we learn in school?
Asked Questions
given Answers

Questions & Answers

® Practice exercises
(training data)
® Mock exams (validation)
® Final exams (testing)

Supervised Learning
 The story behind Deep Learning

Human
Learning from
Intelligence past data to
Exhibited by make accurate
Machines predictions

Source: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/
12
How do
children
learn?

13
“Human beings learn concepts starting from raw data.
For instance, a child shown a few examples/ instances
of a particular animal (like, say, cats) will soon learn to
identify cats.
The learning process does not involve a parent
identifying features (e.g., look at the whiskers or see
the fur or the tail).
Human learning goes from raw data to a conclusion
without the explicit step where features are identified
and provided to the learner.”
- Deep Learning with Python: A Hands-on Introduction by Nikhil Ketkar

® Learn from raw data

14
 The story behind Deep Learning

Human
Learning from
Intelligence past data to Learning from raw
Exhibited by make accurate data
Machines predictions

Source: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/
15
 Outline
• The story behind Deep Learning
• Applications
• Drivers of the recent breakthroughs

16
 27 January 2016

Deep-learning software (AlphaGo) defeats

human Go professional for the first time 17
https://www.youtube.com/watch?v=WXuK6gekU1Y for the award-winning documentary
 Why was AlphaGo‘s Win Such a Big Deal?

IBM chess computer Deep Blue

beat grandmaster Garry Kasparov in 1997
• Go vs Chess
– Bigger board, 19x19 vs 8x8
– Simpler rules, more move possibilities, 361 vs 28
– Longer game, 150 vs 80
• AlphaGo vs Deep Blue
– Deep Blue was programmed to play Chess.
– AlphaGo was a general-purpose algorithm,
applied to win 49 different arcade games
18
Computer Chess (IBM’s Deep Blue vs. Garry Kasparov)
• Feb 1996: 1-2-3 (wins-draws-losses)
• May 1997: 2-3-1
Google: search engine

20
Google image search - “cat and dog”

21
Netflix: movie recommender system

22
Google translate: machine translation

23
Tesla: steering for self-driving cars

24
Art and ….

25
… film-making

26
 Outline
• The story behind Deep Learning
• Applications
• Drivers for the recent breakthroughs

27
Perceptron

1954 1980s - multi-layer

2010’s – the power of
layers
NNs can compute any
function 28
A
Algorithms
Open-source AI software (e.g. Tensorflow, Keras, PyTorch)

B For example,
• ImageNet
Big data • ~15M images collected from web
• Labelled into roughly 22K categories

C
Computing GPUs and cloud resources
power
D
Dollars Google, Facebook, Amazon, Elon Musk…
29