Scribd
Upload
26 views50 pages

10.2. Deep Learning (CNN)

The document is a presentation on Convolutional Neural Networks (CNNs) by Dr. Oybek Eraliev, covering model representation, convolutional and pooling layer explanations, and implementation in TensorFlow. It includes historical context, detailed explanations of convolutional layers, and examples of parameters and outputs. The document emphasizes the importance of hyperparameters in CNN architecture and the role of pooling layers in introducing spatial invariance.

Uploaded by

horaned409
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
26 views50 pages

10.2. Deep Learning (CNN)

The document is a presentation on Convolutional Neural Networks (CNNs) by Dr. Oybek Eraliev, covering model representation, convolutional and pooling layer explanations, and implementation in TensorFlow. It includes historical context, detailed explanations of convolutional layers, and examples of parameters and outputs. The document emphasizes the importance of hyperparameters in CNN architecture and the role of pooling layers in introducing spatial invariance.

Uploaded by

horaned409
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 50

Deep Learning:

Convalutional Neural
Network
Part – 2
Dr. Oybek Eraliev,
Department of Computer Engineering
Inha University In Tashkent.
Email: [email protected]

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 1


Content

ØModel representation of Convolutional Neural Network


ØConvolutional Layer Explanation
ØPooling Layer Explanation
ØImplementation of CNN in TensorFlow Framework

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 2


Model representation of CNN
A bit of history:
ImageNet Classification with Deep Convolutional Neural
Networks [Krizhevsky, Sutskever, Hinton, 2012]

AlexNet
Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 3
Model representation of CNN
A bit of history:
Fast-forward: ConvNets are everywhere Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 4


Model representation of CNN
A bit of history:
Fast-forward: ConvNets are everywhere Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 5


Model representation of CNN
A bit of history:
Fast-forward: ConvNets are everywhere Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 6


Model representation of CNN
A bit of history:
Fast-forward: ConvNets are everywhere Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 7


Model representation of CNN

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 8


Content

ØModel representation of Convolutional Neural Network


ØConvolutional Layer Explanation
ØPooling Layer Explanation
ØImplementation of CNN in TensorFlow Framework

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 9


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 10


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 11


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 12


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 13


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 14


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 15


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 16


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 17


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 18


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 19


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 20


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 21


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 22


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 23


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 24


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 25


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 26


Convolutional Layer Explanation Source: Stenford CS231n

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 27


Convolutional Layer Explanation
A closer look at spatial dimensions:

7x7 input (spatially)


assume 3x3 filter
7

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 28


Convolutional Layer Explanation
A closer look at spatial dimensions:

7x7 input (spatially)


assume 3x3 filter
7

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 29


Convolutional Layer Explanation
A closer look at spatial dimensions:

7x7 input (spatially)


assume 3x3 filter
7

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 30


Convolutional Layer Explanation
A closer look at spatial dimensions:

7x7 input (spatially)


assume 3x3 filter
7

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 31


Convolutional Layer Explanation
A closer look at spatial dimensions:

7x7 input (spatially)


assume 3x3 filter
7
=> 5x5 output

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 32


Convolutional Layer Explanation
A closer look at spatial dimensions:

7x7 input (spatially)


assume 3x3 filter applied with
7 stride 2

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 33


Convolutional Layer Explanation
A closer look at spatial dimensions:

7x7 input (spatially)


assume 3x3 filter applied with
7 stride 2

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 34


Convolutional Layer Explanation
A closer look at spatial dimensions:

7x7 input (spatially)


assume 3x3 filter applied with
7 stride 2

=> 3x3 output!

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 35


Convolutional Layer Explanation
A closer look at spatial dimensions:

7
7x7 input (spatially) assume
3x3 filter applied with stride 3?

7 doesn’t fit!
cannot apply 3x3 filter on 7x7
input with stride 3.

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 36


Convolutional Layer Explanation
In practice: Common to zero pad the border

e.g. input 7x7


3x3 filter, applied with stride 1
pad with 1 pixel border => what is the
output?

(recall:)
(N-F)/stride+1

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 37


Convolutional Layer Explanation
In practice: Common to zero pad the border

e.g. input 7x7


3x3 filter, applied with stride 1
pad with 1 pixel border => what is the output?

7x7 output!
in general, common to see CONV layers with
stride 1, filters of size FxF, and zero-padding
with (F-1)/2. (will preserve size spatially)
e.g.F=3=>zero pad with 1
F=5=>zero pad with 2
F=7=>zero pad with 3

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 38


Convolutional Layer Explanation

Examples time:

Input volume: 32x32x3


10 5x5 filters with stride 1, pad 2

Output volume size: ?

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 39


Convolutional Layer Explanation

Examples time:

Input volume: 32x32x3


10 5x5 filters with stride 1, pad 2

Output volume size: (32+2*2-5)/1+1 = 32


spatially, so 32x32x10

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 40


Convolutional Layer Explanation

Examples time:

Input volume: 32x32x3


10 5x5 filters with stride 1, pad 2

Number of parameters in this layer?

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 41


Convolutional Layer Explanation

Examples time:

Input volume: 32x32x3


10 5x5 filters with stride 1, pad 2

Number of parameters in this layer?


each filter has 5*5*3 + 1 = 76 params => 76*10 = 760

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 42


Convolutional Layer Explanation
Summary
Let’s assume input is W1 x H1 x C Conv layer needs 4 hyperparameters:
• Number of filters K
• The filter size F
• The stride S
• The zero padding P
This will produce an output of W2 x H2 x K where:
• W2 =(W1 -F+2P)/S+1
• H2 =(H1 -F+2P)/S+1

Number of parameters: F2CK and K biases

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 43


Content

ØModel representation of Convolutional Neural Network


ØConvolutional Layer Explanation
ØPooling Layer Explanation
ØImplementation of CNN in TensorFlow Framework

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 44


Pooling Layer Explanation

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 45


Pooling Layer Explanation
Max Pooling

Single depth slice

1 1 2 4 max pool with 2x2 filters


and stride 2
5 6 7 8 6 8

3 2 1 0 3 4
•No learnable parameters
1 2 3 4 •Introduces spatial invariance

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 46


Pooling Layer Explanation
Average Pooling

Single depth slice

1 1 2 4 Average pool with 2x2


filters and stride 2
5 6 7 8 3.25 5.25

3 2 1 0 2 2
•No learnable parameters
1 2 3 4 •Introduces spatial invariance

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 47


Pooling Layer Explanation
Summary

Let’s assume input is W1 x H1 x C Conv layer needs 2 hyperparameters:

• The spatial extent F


• The stride S

This will produce an output of W2 x H2 x C where:

• W2 =(W1 -F)/S+1
• H2 =(H1 -F)/S+1

Number of parameters: 0

Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 48


Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 49
Dr. Oybek Eraliyev Class: Artificial Intelligence SOC4040 50

You might also like