11/11/22, 6:50 AM What is Transposed Convolutional Layer?

| by Aqeel Anwar | Towards Data Science

Get unlimited access Open in app

Published in Towards Data Science

Aqeel Anwar Follow

Mar 6, 2020 · 5 min read · Listen

Save

What is Transposed Convolutional Layer?

Explained through animated gifs and python code
The transposed Convolutional Layer is also (wrongfully) known as the
Deconvolutional layer. A deconvolutional layer reverses the operation of a standard
convolutional layer i.e. if the output generated through a standard convolutional
layer is deconvolved, you get back the original input. The transposed convolutional
layer is similar to the deconvolutional layer in the sense that the spatial dimension
generated by both are the same. Transposed convolution doesn’t reverse the
standard convolution by values, rather by dimensions only.

The transposed convolutional layer does exactly what a standard convolutional layer
does but on a modified input feature map. Before explaining the similarity, let's first
have a look at how does a standard convolutional layer works.

Standard Convolutional Layer:

A standard convolutional layer on an input of size ixi is defined by the following two
parameters.

Padding (p): The number of zeros padded around the original input increasing
the size to (i+2*p)x(i+2*p)

https://towardsdatascience.com/what-is-transposed-convolutional-layer-40e5e6e31c11 1/6
11/11/22, 6:50 AM What is Transposed Convolutional Layer? | by Aqeel Anwar | Towards Data Science

Stride (s): The amount by which the kernel is shifted when sliding across the
Get unlimited access Open in app
input image.

The figure below shows how a convolutional layer works as a two-step process.

In the first step, the input image is padded with zeros, while in the second step the
kernel is placed on the padded input and slid across generating the output pixels as
dot products of the kernel and the overlapped input region. The kernel is slid across
the padded input by taking jumps of size defined by the stride. The convolutional
layer usually does a down-sampling i.e. the spatial dimensions of the output are less
than that of the input.

The animations below explain the working of convolutional layers for different
values of stride and padding.

For a given size of the input (i), kernel (k), padding (p), and stride (s), the size of the
output feature map (o) generated is given by
https://towardsdatascience.com/what-is-transposed-convolutional-layer-40e5e6e31c11 2/6
11/11/22, 6:50 AM What is Transposed Convolutional Layer? | by Aqeel Anwar | Towards Data Science

Get unlimited access Open in app

Transposed Convolutional Layer:

A transposed convolutional layer, on the other hand, is usually carried out for
upsampling i.e. to generate an output feature map that has a spatial dimension
greater than that of the input feature map. Just like the standard convolutional layer,
the transposed convolutional layer is also defined by the padding and stride. These
values of padding and stride are the one that hypothetically was carried out on the
output to generate the input. i.e. if you take the output, and carry out a standard
convolution with stride and padding defined, it will generate the spatial dimension
same as that of the input.

Implementing a transposed convolutional layer can be better explained as a 4 step

process

Step 1: Calculate new parameters z and p’

Step 2: Between each row and columns of the input, insert z number of zeros.
This increases the size of the input to (2*i-1)x(2*i-1)

Step 3: Pad the modified input image with p’ number of zeros

Step 4: Carry out standard convolution on the image generated from step 3 with
a stride length of 1

The complete steps can be seen in the figure below.

Image By Author

The animations below explain the working of convolutional layers for different
values of stride and padding.

https://towardsdatascience.com/what-is-transposed-convolutional-layer-40e5e6e31c11 3/6
11/11/22, 6:50 AM What is Transposed Convolutional Layer? | by Aqeel Anwar | Towards Data Science

Get unlimited access Open in app

For a given size of the input (i), kernel (k), padding (p), and stride (s), the size of the
output feature map (o) generated is given by

Summary:
The table below summarizes the two convolutions, standard and transposed.

The idea behind transposed convolution is to carry out trainable upsampling

Transposed convolutions are standard convolutions but with a modified input

feature map.

The stride and padding do not correspond to the number of zeros added around
the image and the amount of shift in the kernel when sliding it across the input,
as they would in a standard convolution operation.

Python Code:

https://towardsdatascience.com/what-is-transposed-convolutional-layer-40e5e6e31c11 4/6
11/11/22, 6:50 AM What is Transposed Convolutional Layer? | by Aqeel Anwar | Towards Data Science

The gifs were generated using python. The complete code can be found at
Get unlimited access Open in app
https://github.com/aqeelanwar/conv_layers_animation

Bonus:
Compact cheat sheets for this topic and many other important topics in Machine
Learning can be found in the link below

Cheat Sheets for Machine Learning Interview Topics

A visual cheatsheet for ML interviews (www.cheatsheets.aqeel-
anwar.com)
medium.com

If this article was helpful to you, feel free to clap, share and respond to it. If you
want to learn more about Machine Learning and Data Science, follow me @
Aqeel Anwar or connect with me on LinkedIn.

304 7

Sign up for The Variable

By Towards Data Science

Every Thursday, the Variable delivers the very best of Towards Data Science: from hands-on tutorials and cutting-
edge research to original features you don't want to miss. Take a look.

Emails will be sent to [email protected]. Not you?

Get this newsletter

https://towardsdatascience.com/what-is-transposed-convolutional-layer-40e5e6e31c11 5/6
11/11/22, 6:50 AM What is Transposed Convolutional Layer? | by Aqeel Anwar | Towards Data Science

Get unlimited access Open in app

https://towardsdatascience.com/what-is-transposed-convolutional-layer-40e5e6e31c11 6/6