Scribd
Upload
7 views33 pages

Lecture 3 - Pretrained CNN - CET

Uploaded by

Joel Lim
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
7 views33 pages

Lecture 3 - Pretrained CNN - CET

Uploaded by

Joel Lim
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 33

Official (Closed) - Non Sensitive

Deep Learning in Image


Recognition
Lecture 3:
Pretrained Convolutional
Neural Networks

• Specialist Diploma in Applied


Generative AI

• Academic Year 2024/25


Official (Closed) - Non Sensitive

Topics

1. Using a pre-trained CNN

2. Feature Extraction without Data


Augmentation

3. Feature Extraction with Data Augmentation

4. Fine-tuning a pre-trained CNN

5. Going beyond Sequential model


Official (Closed) - Non Sensitive

1. Using a pre-trained CNN


Official (Closed) - Non Sensitive

Using a pre-trained CNN


 Common and highly effective to use a
pre-trained network
 Pre-trained network is a saved network
what was previously trained on a large
dataset
 The spatial hierarchy of features learned
can effectively act as a generic model
 The portability of learned features
across different problems is a key
advantage of deep learning vs. older,
shallow-learning approaches
Official (Closed) - Non Sensitive

Using a pre-trained CNN


 We will use a large CNN trained on the
ImageNet dataset, which has:
1. 1.4 million labeled images and 1,000
different classes
2. Many animal classes, including different
species of cats and dogs

 The well-known image processing


models:
1. VGG 16 (we will use this one) and VGG 19
2. Inception V3, ResNet50, Xception,
MobileNet
3. All available in keras.applications
Official (Closed) - Non Sensitive

2. Feature Extraction without Data


Augmentation
Official (Closed) - Non Sensitive

Overview
 Our own classifier vs VGG16 classifer

Our own:
Cat & Dog classifier
(2 classes)

VGG model:
FC classifier
1000 classes
Official (Closed) - Non Sensitive

Overview
 Objective: use VGG16 model and train our own FC
classifier
Conv block 1 Conv block 2 Conv block 3 Conv block 4 Conv block 5
(frozen) (frozen) (frozen) (frozen) (frozen)

Conv_Base = VGG16 – FC Classifier FC Classifier


Official (Closed) - Non Sensitive

Overview
 Objective: use VGG16 model and train our own FC
classifier

FC Classifier

Conv_Base = VGG16 – FC Classifier


https://www.mygreatlearning.com/blog/introduction-to-vgg16/
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image
recognition. In ICLR, 2015.
Official (Closed) - Non Sensitive

Step 1: Download VGG16 model as


conv_base
 Download pre-trained VGG16 model

 Exclude the last few fully connected layers


(“classifier”) from VGG16

 Train only our own fully connected (FC) classifier


for final classification
Official (Closed) - Non Sensitive

Step 2: Extract features from sample images


by calling predict() method of the conv_base
mode

Inputs: Outputs:
Training Images conv_base Training Features
(VGG16)
Validation Images Validation Features
(150 * 150 * 3 (RGB) size) (# samples, 4,4,512 )

Neural Network Forward Propagation


Official (Closed) - Non Sensitive

Step 3: Train our own fully


connected classifier
Output from Step 2

Inputs: Our Own


Training Features FC Classifier Outputs:
Validation Features Probability of Cat / Dog
(after flatten out)

Model is trained very quickly


Official (Closed) - Non Sensitive

Let’s try your hands on your first pre-trained


model!
(Practical 3a - Part 1.1: Feature Extraction without
Data Augmentation)
Official (Closed) - Non Sensitive

Result
Official (Closed) - Non Sensitive

3. Feature Extraction with Data


Augmentation
Official (Closed) - Non Sensitive

 Training the whole Model with VGG16


layers frozen
• Much slower and more expensive (GPU
support preferred)
 Allow to use data augmentation during
Trainable
training
Conv block 1 Conv block 2 Conv block 3 Conv block 4 Conv block 5 layers
(frozen) (frozen) (frozen) (frozen) (frozen)
• avoid overfitting

Our Own Fully


Connected Classify
VGG16
Official (Closed) - Non Sensitive

Let’s try your hands on your first pre-trained


model!
(Practical 3a - Part 1.2: Feature Extraction with
Data Augmentation)
Official (Closed) - Non Sensitive

 Results
Official (Closed) - Non Sensitive

4. Fine-Tuning a Pre-trained CNN


Official (Closed) - Non Sensitive

 Training the whole model with some of


VGG16 layers trainable (unfrozen)
1. Conv block 5 (4 layers) is trainable

Trainable layers
Conv block 1 Conv block 2 Conv block 3 Conv block 4 Conv block 5
(frozen) (frozen) (frozen) (frozen) (Trainable)

Our Own
VGG16 (conv_base) FC classifier
Official (Closed) - Non Sensitive

 Steps

1. Add your own fully connected classifier on


top of an already-trained base network
(VGG16 – top)

2. Freeze the base network

3. Train the layers you added

4. Unfreeze some layers in the base network

5. Jointly train both these layers and the


layers you added
Official (Closed) - Non Sensitive

Let’s try your hands on your first pre-trained


model!
(Practical 3a - Part II: Fine Tuning a pre-trained
model)
Official (Closed) - Non Sensitive

 Results
Official (Closed) - Non Sensitive

Video Source:
https://www.youtube.com/watch?v=H8sXcAXrGR4
(~28 minutes)
Official (Closed) - Non Sensitive

5. Going beyond
Sequential model
Official (Closed) - Non Sensitive

 Going beyond
Sequential model

Xception CNN
Official (Closed) - Non Sensitive

Practical 3: Keras functional API


 Build a question-
answering Model
1. Two inputs
 Question
 Text: info to answer
Q
2. Merge the two
input branches
using Concatenate
layer
3. One Output
 Answer
Official (Closed) - Non Sensitive

Let’s try your hands on your first pre-trained


model!
(Practical 3b – Keras Functional API)
Official (Closed) - Non Sensitive

Wrapping up
 It’s very helpful to use
feature extraction to reuse a
pre-trained convnet on a new
dataset

 You can also use fine-tuning


on a pre-trained CNN as this
pushes the model’s
performance even further
Official (Closed) - Non Sensitive

Assignment Briefing and


presentation time slot
 Spreadsheet

 Assignment consultation – Wk 7
Official (Closed) - Non Sensitive

Further Reading
 Step by step VGG16 implementation in Keras for
beginners
• https://towardsdatascience.com/step-by-step-vgg16-i
mplementation-in-keras-for-beginners-a833c686ae6c

 Transfer Learning from Pre-trained Models


• https://towardsdatascience.com/transfer-learning-fro
m-pre-trained-models-f2393f124751

 How to Add Regularization to Keras Pre-trained


Models the Right Way
• https://towardsdatascience.com/how-to-add-regulariz
ation-to-keras-pre-trained-models-the-right-way-7437
76451091
Official (Closed) - Non Sensitive

Q&A
Official (Closed) - Non Sensitive

References
Books:

François Chollet, Deep Learning with Python (2018)

Online Resources:

You might also like