Scribd
Upload
72 views20 pages

Lecture 26-30 Unit 2

Uploaded by

topendrabdr1
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
72 views20 pages

Lecture 26-30 Unit 2

Uploaded by

topendrabdr1
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 20

APEX INSTITUTE OF TECHNOLOGY

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

DEEP LEARNING (20CSF-432)


Faculty: Dr. Amit Kukker(E16298)

Lecture – 26-30
Training a CNNs: weights initialization, batch DISCOVER . LEARN . EMPOWER
normalization, hyper parameter optimization,
By: Dr. Amit Kukker 1
Deep Learning: Course Objectives
COURSE OBJECTIVES
The Course aims to:
1. Understand the key features in a neural network’s architecture
2. Understand the main fundamentals that drive Deep Learning
3. Be able to build, train and apply fully connected deep neural networks
4. Know how to implement efficient CNN, LSTM, Bi-LSTM, Autoencoder, RNN, Adversarial
Generative Networks etc.
5. Implementation the fundamental methods involved in deep learning, including the underlying
optimization concepts (gradient descent and backpropagation) and how they can be combined to
solve real-world problems.

By: Dr. Amit Kukker 2


COURSE OUTCOMES

On completion of this course, the students shall be able to:-


CO1 Understand neural network, its working and parameters, and various optimization methods for
neural networks.
CO2 Differentiate between the major types of neural network architectures and its use case for
different problems (classification/recognition) by these architectures.

CO3 Understand different deep neural network model architectures and its parameters tuning.

CO4 Design sequence model using different neural network architectures for new data problems based
on their requirements and problem characteristics and analyse their performance.

CO5 Describe latest research being conducted in the field and open problems that are yet to be solved.

By: Dr. Amit Kukker 3


Unit-2 Syllabus
Unit-2 Second Order Methods

Second order methods for training,


Regularization methods (dropout, drop connect,
batch normalization), Introduction to CNNs -
Second Order convolution, pooling, Deep CNNs, Different deep
Methods CNN architectures - LeNet, AlexNet, VGG,
Training a CNNs: weights initialization, batch
normalization, hyper parameter optimization,
Understanding and visualizing CNNs

By: Dr. Amit Kukker 4


SUGGESTIVE READINGS
TEXT BOOKS:
• T1: Deep Learning with Python by Francois Chollet, Publisher: Manning Publications
• T2: Deep Learning from Scratch: Building with Python from First Principles by Seth Weidman
published by O`Reilley
• T3: Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville published by MIT Press.

REFERENCE BOOKS:
• R1 Fundamentals of Deep Learning: by Nithin Buduma, Nikhil Buduma and Joe Papa, OREILLY
Publication, Second Edition.
• R2 Deep Learning: A Practitioners Approach by Josh Patterson and Adam Gibson, OREILLY
Publication.
• R3 Deep Learning for Coders with fastai and PyTorch by Jeremy Howard and Sylvain Gugger, OREILLY
Publication.
• R4 Deep Learning Using Python by S Lovelyn Rose, L Ashok Kumar, D Karthika Renuka, Wiley
Publication
By: Dr. Amit Kukker 5
Training a CNNs

By: Dr. Amit Kukker 6


def xavier_init(shape):
fan_in, fan_out = shape
limit = np.sqrt(6 / (fan_in + fan_out))
return np.random.uniform(-limit, limit, shape)

def he_init(shape):
fan_in, _ = shape
std = np.sqrt(2.0 / fan_in)
return np.random.randn(*shape) * std

By: Dr. Amit Kukker 7


By: Dr. Amit Kukker 8
By: Dr. Amit Kukker 9
By: Dr. Amit Kukker 10
By: Dr. Amit Kukker 11
from sklearn.model_selection import GridSearchCV
parameters = {'learning_rate': [0.1, 0.01, 0.001], 'batch_size': [16, 32, 64]}

from sklearn.model_selection import RandomizedSearchCV


parameters = {'learning_rate': [0.1, 0.01, 0.001], 'batch_size': [16, 32, 64]}

By: Dr. Amit Kukker 12


4. Understanding and Visualizing CNNs

Understanding and visualizing CNNs help in diagnosing and


improving model performance.

By: Dr. Amit Kukker 13


import matplotlib.pyplot as plt

def visualize_filters(layer, n_filters=6):


filters = layer.weight.data.clone()
filters = filters - filters.min()
filters = filters / filters.max()
filters = filters.numpy()

fig, ax = plt.subplots(1, n_filters, figsize=(20, 5))


for i in range(n_filters):
ax[i].imshow(filters[i, 0, :, :], cmap='gray')
ax[i].axis('off')
plt.show()

visualize_filters(model.conv1, n_filters=6)
By: Dr. Amit Kukker 14
def visualize_feature_maps(model, image):
x = image.unsqueeze(0) # Add batch dimension
layers = [model.conv1, model.bn1, model.conv2, model.bn2]
fig, ax = plt.subplots(len(layers), 1, figsize=(20, 20))

for i, layer in enumerate(layers):


x = layer(x)
if isinstance(layer, nn.BatchNorm2d):
x = torch.relu(x)
x = torch.max_pool2d(x, 2)
ax[i].imshow(x[0, 0, :, :].detach().numpy(), cmap='gray')
ax[i].axis('off')
plt.show()

visualize_feature_maps(model, example_image)
By: Dr. Amit Kukker 15
def saliency_map(model, image, label):
image.requires_grad_()
output = model(image)
loss = nn.CrossEntropyLoss()(output, label)
loss.backward()

saliency, _ = torch.max(image.grad.data.abs(), dim=1)


saliency = saliency.squeeze().cpu().numpy()

plt.imshow(saliency, cmap='hot')
plt.axis('off')
plt.show()

saliency_map(model, example_image.unsqueeze(0), example_label.unsqueeze(0))

By: Dr. Amit Kukker 16


By: Dr. Amit Kukker 17
References
Main text books:
• “Neural Networks: A Comprehensive Foundation”, S. Haykin (very
good -theoretical)
• “Pattern Recognition with Neural Networks”, C. Bishop (very good accessible)
• “Neural Network Design” by Hagan, Demuth and Beale (introductory)
• Books emphasizing the practical aspects:
• “Neural Smithing”, Reeds and Marks
• “Practical Neural Network Recipees in C++”’ T. Masters
• Seminal Paper (but now quite old!):
“Parallel Distributed Processing” Rumelhart and McClelland et al.
Deep Learning books and tutorials:
• http://www.deeplearningbook.org/
• Introduction to Learning Rules in Neural Network - DataFlair (data-flair.training) 18
Neural Networks Literature
Review Articles:
• R. P. Lippman, “An introduction to Computing with Neural Nets”’ IEEE
• ASP Magazine, 4-22, April 1987.
• T. Kohonen, “An Introduction to Neural Computing”, Neural Networks,
• 1, 3-16, 1988.
• A. K. Jain, J. Mao, K. Mohuiddin, “Artificial Neural Networks: A Tutorial”’
• IEEE Computer, March 1996’ p. 31-44.
Journals:
• IEEE Transactions on NN
• Neural Networks
• Neural Computation
• Biological Cybernetics
19
THANK YOU

For queries
Email: [email protected]

You might also like