Scribd
Upload
37 views10 pages

2017 - Lecture 5 - Smaller Network - CNN - 2 (Ming Li) (10 Slides)

This document compares and contrasts convolutional neural networks (CNNs) and fully connected neural networks. CNNs reduce the number of parameters by using local connections and shared weights between layers. They also use max pooling to downsample images while preserving important information. The overall CNN architecture alternates between convolutional and max pooling layers to create progressively smaller representations, followed by fully connected layers for classification.

Uploaded by

rvasiliou
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
37 views10 pages

2017 - Lecture 5 - Smaller Network - CNN - 2 (Ming Li) (10 Slides)

This document compares and contrasts convolutional neural networks (CNNs) and fully connected neural networks. CNNs reduce the number of parameters by using local connections and shared weights between layers. They also use max pooling to downsample images while preserving important information. The overall CNN architecture alternates between convolutional and max pooling layers to create progressively smaller representations, followed by fully connected layers for classification.

Uploaded by

rvasiliou
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 10

Convolution v.s.

Fully Connected

1 0 0 0 0 1 1 -1 -1 -1 1 -1
0 1 0 0 1 0 -1 1 -1 -1 1 -1
0 0 1 1 0 0 -1 -1 1 -1 1 -1
1 0 0 0 1 0
0 1 0 0 1 0
0 0 1 0 1 0
convolution
image

x1
1 0 0 0 0 1
0 1 0 0 1 0 x2
Fully- 0 0 1 1 0 0
1 0 0 0 1 0
connected
…
…

…
…
0 1 0 0 1 0
0 0 1 0 1 0
x36
1 -1 -1 Filter 1 1 1
-1 1 -1 2 0
-1 -1 1 3 0
4 0 3
:

…
1 0 0 0 0 1
0 1 0 0 1 0 0
0 0 1 1 0 0 8 1
1 0 0 0 1 0 9 0
0 1 0 0 1 0 10: 0

…
0 0 1 0 1 0
13 0
6 x 6 image
14 0
fewer parameters! 15 1 Only connect to
16 1 9 inputs, not
fully connected
…
1 -1 -1 1 1
:2 0
-1 1 -1 Filter 1
:3 0
-1 -1 1
:4 0 3
:

…
1 0 0 0 0 1
0 1 0 0 1 0 7 0
:8 1
0 0 1 1 0 0
:9 0 -1
1 0 0 0 1 0
10:: 0
0 1 0 0 1 0

…
0 0 1 0 1 0
13 0
6 x 6 image
:
14 0
Fewer parameters :15 1
:
16 1 Shared weights
Even fewer parameters
:
…
The whole CNN
cat dog ……
Convolution

Max Pooling
Can repeat
Fully Connected many
Feedforward network
Convolution times

Max Pooling

Flattened
Max Pooling
1 -1 -1 -1 1 -1
-1 1 -1 Filter 1 -1 1 -1 Filter 2
-1 -1 1 -1 1 -1

3 -1 -3 -1 -1 -1 -1 -1

-3 1 0 -3 -1 -1 -2 1

-3 -3 0 1 -1 -1 -2 1

3 -2 -2 -1 -1 0 -4 3
Why Pooling

 Subsampling pixels will not change the object


bird
bird

Subsampling

We can subsample the pixels to make image smaller


fewer parameters to characterize the image
A CNN compresses a fully connected
network in two ways:
 Reducing number of connections
 Shared weights on the edges
 Max pooling further reduces the complexity
Max Pooling

New image
1 0 0 0 0 1 but smaller
0 1 0 0 1 0 Conv
3 0
0 0 1 1 0 0 -1 1
1 0 0 0 1 0
0 1 0 0 1 0 Max 30 13
0 0 1 0 1 0 Pooling
2 x 2 image
6 x 6 image
Each filter
is a channel
The whole CNN
3 0
-1 1 Convolution

3 1
0 3
Max Pooling
Can repeat
A new image
many
Convolution times
Smaller than the original
image
The number of channels Max Pooling

is the number of filters


The whole CNN
cat dog ……
Convolution

Max Pooling

Fully Connected A new image


Feedforward network
Convolution

Max Pooling

Flattened A new image

You might also like