Official (Closed) - Non Sensitive
Deep Learning in Image
Recognition
Lecture 4:
Visualizing What Convnets
Learn
• Specialist Diploma in Applied
Generative AI
• Academic Year 2024/25
Official (Closed) - Non Sensitive
Topics
1. Introduction to Visualizing
Convnets
2. Activation Visualization
3. Visualizing Convnets Filters
4. Visualizing Heatmaps of Class
Activation
Official (Closed) - Non Sensitive
1. Introduction to Visualizing Convnets
Official (Closed) - Non Sensitive
Deep Learning Models as Black
Boxes
Deep-learning models as black boxes
• Learning representations that are hard to
extract
• Not true for convnets
Convnets and visual concepts
• Representations are highly amenable to
visualization
• Representations of visual concepts
Techniques for visualization and
interpretation
• Wide array of techniques since 2013
• Three accessible and useful techniques
Official (Closed) - Non Sensitive
Techniques for Visualizing
Convnets
1. Visualizing intermediate
convnet outputs
• Helps understand how
successive convnet layers
transform their input
• Provides insight into the
meaning of individual convnet
filters
Official (Closed) - Non Sensitive
Techniques for Visualizing
Convnets
2. Visualizing convnet filters
• Helps understand what visual
pattern or concept each filter is
receptive to
3. Visualizing heatmaps of class
activation in an image
• Helps understand which parts of
an image were identified as
belonging to a given class
• Allows localization of objects in
images
Official (Closed) - Non Sensitive
2. Activation Visualization
Official (Closed) - Non Sensitive
What is intermediate activation
Intermediate activations are the outputs
generated by each layer in a Convnet when
an image is passed through it
Official (Closed) - Non Sensitive
Why Visualize intermediate
activation
Ability to understand what the model
“sees” or “focuses on” at each layer
Useful for debugging and interpreting
model behavior
Able to provide explainability to users
Official (Closed) - Non Sensitive
Steps to Visualize
Step 1:
• Load the model and prepare the image
Step 2:
• Create a model that outputs layer activations
Step 3:
• Generate activations for an input image
Step 4:
• Visualize the activations
Official (Closed) - Non Sensitive
Let’s try your hands on visualizing our model!
(Practical 4 - Part 1: Visualizing intermediate
activation)
Official (Closed) - Non Sensitive
What was seen
Official (Closed) - Non Sensitive
3. Visualizing Convnets Filters
Official (Closed) - Non Sensitive
What is filter visualization
A technique to understand and interpret
what individual filter in a convnet are
responding to in input images
Involves visualizing the patterns or
features that activate each filter
Official (Closed) - Non Sensitive
Why filter visualization
Interpretability: It helps us understand the inner
workings of convnets by showing us what each
filter detects in an image
Debugging: By examining filters, we can diagnose
potential issues, such as overfitting or filters not
learning meaningful patterns
Feature Understanding: Visualization highlights
the progression from low-level to high-level feature
detection as we move deeper into the network
Official (Closed) - Non Sensitive
How filter visualization works
Filter visualization is generally achieved by
maximizing the activation of each filter, using
techniques like gradient ascent
This process essentially creates synthetic
images that show which types of patterns cause
a high response in a particular filter
These synthetic images provide a visual
representation of the filter’s learned features
Official (Closed) - Non Sensitive
Let’s try your hands on visualizing our model!
(Practical 4 - Part 2: Visualizing convnet filters)
Official (Closed) - Non Sensitive
How filter looks like
Official (Closed) - Non Sensitive
4. Visualizing Heatmaps of Class
Activation
Official (Closed) - Non Sensitive
Introduction to Class Activation
Maps
Understanding Image Classification Decisions
• Useful for debugging convnet decisions
• Helps locate specific objects in an image
Class Activation Map (CAM) Visualization
• Produces heatmaps of class activation over input images
• 2D grid of scores for a specific output class
• Indicates importance of each location in the image
Application Example
• Dogs vs. Cats convnet
• Heatmap for class
Official (Closed) - Non Sensitive
Gradient Class Activation Maps
(Grad-CAM)
Uses output feature map of a convolution layer
Weigh every channel in the feature map by the
gradient of the class with respect to the channel
Create a spatial map of how intensely the input
image activates different channels
Ramprasaath R. Selvaraju et al., arXiv (2017), https://arxiv.org/abs/ 1610.02391
Official (Closed) - Non Sensitive
Let’s try your hands on visualizing our model!
(Practical 4 - Part 3: Visualizing heatmaps of class
activation in an image)
Official (Closed) - Non Sensitive
What Grad-CAM generated
Ramprasaath R. Selvaraju et al., arXiv (2017), https://arxiv.org/abs/ 1610.02391
Official (Closed) - Non Sensitive
Wrapping up
Understanding ConvNet
learning and representation
Visualization techniques
applied on activation, filter
and heatmaps
Understand model focus to
provide interpretability
Official (Closed) - Non Sensitive
Q&A
Official (Closed) - Non Sensitive
References
Books:
François Chollet, Deep Learning with Python
Online Resources:
