Deep

Learning
James Hinns
My Background
• BSc Computer Science
• MSc by Research – XAI
• PhD Multi-Modal Explanations – Prof. David Martens

Deep Learning - Case Studies and Trends in Data Science 22-23 2

Introduction

Deep Learning - Case Studies and Trends in Data Science 22-23 3

Deep Learning - Case Studies and Trends in Data Science 22-23 4
Ever growing public
interest

Deep Learning - Case Studies and Trends in Data Science 22-23 5

What is Deep Learning?

Artificial
Intelligence Machine
Any technique that Learning
enables computers to
Ability to learn without
Deep
mimic human behaviour
being explicitly Learning
programmed. Enables
Extract patterns from
machines to improve at
data using neural
tasks with experience.
networks

Deep Learning - Case Studies and Trends in Data Science 22-23 6

What is Deep Learning?

Deep Learning - Case Studies and Trends in Data Science 22-23 7

Feature Engineering

M2 Rental Cost M2 Rental Cost £/M2

100 £960.00 100 £960.00 £9.60
60 £785.00 60 £785.00 £13.08
87 £845.00 87 £845.00 £9.71
54 £650.00 54 £650.00 £12.04
92 £1,200.00 92 £1,200.00 £13.04

Deep Learning - Case Studies and Trends in Data Science 22-23 8

Feature Engineering

Low Level Features Mid Level Features High Level Features

Lines and Edges Eyes, Noses, Ears Facial Structure

Taken from: http://introtodeeplearning.com/slides/6S191_MIT_DeepLearning_L1.pdf

Deep Learning - Case Studies and Trends in Data Science 22-23 9

Why use deep learning?
“The analogy to deep
learning is that the rocket
engine is the deep learning
models and the fuel is the
huge amounts of data we
can feed to these
algorithms.” – Andrew Ng

Image source: https://towardsdatascience.com/why-deep-learning-is-needed-over-

traditional-machine-learning-1b6a99177063

Deep Learning - Case Studies and Trends in Data Science 22-23 10

 Timeline
1952 Gradient Descent
Big Data
• As discussed
1958 Perceptron

GPUs
1986 Backpropagation
• Parallelisation
1995 CNN

Software
2020 GPT-3

Deep Learning - Case Studies and Trends in Data Science 22-23 11

What we’re covering
• Multi-Layer Perceptron (MLP) recap
• Neural Network Architectures
• Convolutional Neural Networks (CNNs)
• Generative Adversarial Networks (GANs)
• Transformers
• Applications to research

Deep Learning - Case Studies and Trends in Data Science 22-23 12

Resources
• This lecture is a (extremely) brief introduction
• For a nice introduction see:
• MIT 6.S191
• 3Blue1Brown Neural Net Series
• StatQuest Neural Networks Series
• Deep Learning – Goodfellow, Bengio, Courville -
https://www.deeplearningbook.org/ (or search for a pdf)

Deep Learning - Case Studies and Trends in Data Science 22-23 13

Recap: MLPs

M2

M2 Rental Cost £/M2

100 £960.00 £9.60
60 £785.00 £13.08 £
87 £845.00 £9.71
54 £650.00 £12.04
92 £1,200.00 £13.04
£/
M2

Deep Learning - Case Studies and Trends in Data Science 22-23 14

CNNs – classify image with normal neural net

Deep Learning - Case Studies and Trends in Data Science 22-23 15

CNNs – classify image with normal neural net

Deep Learning - Case Studies and Trends in Data Science 22-23 15

 CNNs – classify image with normal neural net
• Could have a NN where each input node is
a pixel.

• Each input node needs to be connected to

every node in the hidden layer.

• Possible with a small 6x6 image.

• In most modern setting this is infeasible.

Images source: https://www.youtube.com/watch?v=HGwBXDKFk9I

Deep Learning - Case Studies and Trends in Data Science 22-23 16

 CNNs – classify image with normal neural net
• Could have a NN where each input node is
a pixel.

• Each input node needs to be connected to

every node in the hidden layer.

• Possible with a small 6x6 image.

• In most modern setting this is infeasible.

Images source: https://www.youtube.com/watch?v=HGwBXDKFk9I

Deep Learning - Case Studies and Trends in Data Science 22-23 16

CNNs – Convolutions – Kernels

Image source: https://en.wikipedia.org/wiki/Kernel_(image_processing)

Deep Learning - Case Studies and Trends in Data Science 22-23 17

CNNs – Convolutions – Kernels

Image source:
https://commons.wikimedia.org/wiki/File:Banana_(white_background).jpg

Deep Learning - Case Studies and Trends in Data Science 22-23 18

CNNs – Creating a feature map
Input Image Kernel Feature Map

Images source: https://www.youtube.com/watch?v=HGwBXDKFk9I

Deep Learning - Case Studies and Trends in Data Science 22-23 19

CNNs – Creating a feature map
Input Image Kernel Feature Map

Images source: https://www.youtube.com/watch?v=HGwBXDKFk9I

Deep Learning - Case Studies and Trends in Data Science 22-23 19

CNNs –
Pooling
Feature Map Post Max
ReLU Pooled
1 0

0 1

Images source: https://www.youtube.com/watch?v=HGwBXDKFk9I

Deep Learning - Case Studies and Trends in Data Science 22-23 20

CNNs – Takeaway
• Reduce number of input nodes.
• Tolerate small shifts in pixel location.
• Take advantage of correlations in images.

Deep Learning - Case Studies and Trends in Data Science 22-23 21

CNN – MNIST Example

Image source: https://machinelearningmastery.com/how-to-develop-a-convolutional-

neural-network-from-scratch-for-mnist-handwritten-digit-classification/

Deep Learning - Case Studies and Trends in Data Science 22-23 22

CNN MNIST Example
• Simple tensorflow code achieving 98% accuracy.

• ChatGPT can write this code for you:

• Ask “Write a CNN to classify the MNIST dataset in tensorflow”

Deep Learning - Case Studies and Trends in Data Science 22-23 23

GANs – Generative Adversarial Networks
Standard Supervised Learning Process

Truth

Input Model Output Evaluate

Deep Learning - Case Studies and Trends in Data Science 22-23 24

GANs – Generative Adversarial Networks

Generator Discriminator

Deep Learning - Case Studies and Trends in Data Science 22-23 25

GANs – Generative Adversarial
Networks
Generator Fake Sample

Real/Fake
Discriminator
Classification

Real Sample

Deep Learning - Case Studies and Trends in Data Science 22-23 26

GANs – Why
use?

Image source: https://arxiv.org/pdf/1703.10593.pdf

Deep Learning - Case Studies and Trends in Data Science 22-23 27

GANs – StyleGAN
• https://thisperson
doesnotexist.com/

Deep Learning - Case Studies and Trends in Data Science 22-23 28

GANs – Mode Image source: https://arxiv.org/pdf/1611.02163.pdf

collapse

Deep Learning - Case Studies and Trends in Data Science 22-23 29

GANs – Image source: https://hardikbansal.github.io/CycleGANBlog/

CycleGAN

Deep Learning - Case Studies and Trends in Data Science 22-23 30

GANs – CycleGAN

Image source: https://junyanz.github.io/CycleGAN/

Deep Learning - Case Studies and Trends in Data Science 22-23 31

Transformers
• Already covered CNNs and GANs
• Were SOTA for a time
• Great for specific purposes

• Like CNNs, they’re efficient, and parallelisable

• GPT3, 175 Billion Parameters, 45TB of Training Data
• LLM SOTA increases in size by a factor of 10 year on year

Deep Learning - Case Studies and Trends in Data Science 22-23 32

Self-Attention

• Fundamentally: What part to focus on?

• Consider language translation of English  French

• The big red dog  Le gros chien rouge

Deep Learning - Case Studies and Trends in Data Science 22-23 33

Self-Attention
Attention Attention
Vectors Vectors
[0.71, 0.04, [1.00, 0.00,
The Le
0.07, 0.18] 0.00, 0.00]

[0.01, 0.84, [0.10, 0.90,

big gros
0.02, 0.13] 0.00, 0.00]

[0.09, 0.05, [0.05, 0.40,

red chien
0.62, 0.24] 0.55, 0.00]

[0.03, 0.03, [0.16, 0.09,

dog rouge
0.03, 0.91] 0.15, 0.66]

Deep Learning - Case Studies and Trends in Data Science 22-23 34

Transformers – Domination of
SOTA
• Transformers are the go to for language tasks
• Chat GPT, GPT3/2/1, Bert etc…

• Transformers also heavily used in image tasks

• Image transformer – Late 2020
• Combination of CNN and Transformers

Deep Learning - Case Studies and Trends in Data Science 22-23 35

Transformers – Domination of
SOTA Semantic Segmentation

Highlighted datasets are

where the SOTA is NOT a
transformer.

Deep Learning - Case Studies and Trends in Data Science 22-23 36

Transformers – Domination of
SOTA Image Classification

Highlighted datasets are

where the SOTA is NOT a
transformer.

Deep Learning - Case Studies and Trends in Data Science 22-23 37

Deep Learning - Case Studies and Trends in Data Science 22-23 38
Research Example
• Producing counterfactual
explanations for images.

Image source: https://link.springer.com/content/pdf/10.1007/s10044-021-01055-y.pdf

Deep Learning - Case Studies and Trends in Data Science 22-23 39
Research Example
• Producing counterfactual
explanations for images.

• Previous methods used filter

based segmentation.

Image source: https://link.springer.com/content/pdf/10.1007/s10044-021-01055-y.pdf

Deep Learning - Case Studies and Trends in Data Science 22-23 39
Research Example
• Instead use Deep Learning to
segment instances within
images.

Image source: https://colab.research.google.com/github/NielsRogge/Transformers-

Tutorials/blob/master/DETR/DETR_panoptic_segmentation_minimal_example_(with_DetrFeatureExtractor).ipynb

Deep Learning - Case Studies and Trends in Data Science 22-23 40

Research Example
• Instead use Deep Learning to
segment instances within
images.

Image source: https://colab.research.google.com/github/NielsRogge/Transformers-

Tutorials/blob/master/DETR/DETR_panoptic_segmentation_minimal_example_(with_DetrFeatureExtractor).ipynb

Deep Learning - Case Studies and Trends in Data Science 22-23 40

This is possible on normal hardware
• Google Colab

• Simple instillation and use on any pc/mac with Python.

Deep Learning - Case Studies and Trends in Data Science 22-23 41

Masters Thesis – Multi-Modal Explanations

Deep Learning - Case Studies and Trends in Data Science 22-23 42

Masters Thesis – Multi-Modal Explanations

Deep Learning - Case Studies and Trends in Data Science 22-23 42

A few more resources
• Two Minute Papers
• The Batch
• Hugging Face
• Medium

Deep Learning - Case Studies and Trends in Data Science 22-23 43

 Questions?

Contact Me:
[email protected]
Deep Learning - Case Studies and Trends in Data Science 22-23 44