Pad Input Tensor Boundaries with Zero in PyTorch

The torch.nn.ZeroPad2D() pads the input tensor boundaries with zeros. It takes the size of padding (padding) as a parameter. The size of padding may be an integer or a tuple. The padding may be the same for all boundaries or different for each boundary.

The padding may be an integer or a tuple in (left, right, top, bottom) format. If it is an integer, then the padding along all the boundaries are the same. The height of the padded tensor is increased by top+bottom, whereas the width of the padded tensor is increased by left+right. It does not change the channel size or batch size. Padding is generally used in convolutional neural networks (CNNs) after Pooling layers to maintain the input size

Syntax

torch.nn.ZeroPad2D(padding)

Parameters

• padding – Desired size of padding. An integer or a tuple in (left, right, top, bottom) format.

The size of the input tensor must be in 3D or 4D in (C, H, W) or (N, C, H, W) format respectively where N, C, H, W represent the mini batch size, number of channels, height and width, respectively.

Steps

We could use the following steps to pad the input tensor boundaries with zero −

• Import the required library. In all the following examples, the required Python library is torch. Make sure you have already installed it.

import torch

• Define the input tensor. We define a 4D tensor as below.

input = torch.randn(2, 1, 3, 3)

• Define a padding size and pass it to torch.nn.ZeroPad2D() and create an instance pad to pad the tensor with zeros. The padding size may be the same or different padding size.

padding = (2,1)
pad = nn.ZeroPad2d(padding)

• Pad the input tensor with zeros using the above created instance pad.

output = pad(input)

• Print the final padded tensor.

print("Padded Ternsor:
", output)

Example 1

In the following Python example, we pad 3D and 4D tensors with zeros using integer padding size of 2, i.e., padding=2

# Import the required library
import torch
import torch.nn as nn

# define 3D tensor (C,H,W)
input = torch.tensor([[[ 1, 2],[ 3, 4]]])
print("Input Tensor:
",input)

# define padding same for all sides (left, right, top, bottom)
pad = nn.ZeroPad2d(2)

# pad the input tensor
output = pad(input)
print("Padded Ternsor:
", output)

# define 4D tensor (N,C,H,W)->for a batch of N tensors
input = torch.randn(2, 1, 3, 3)
print("Input Tensor:
",input)

# define padding same for all sides (left, right, top, bottom)
pad = nn.ZeroPad2d(2)

# pad the input tensor
output = pad(input)
print("Padded Tensor:
", output)

Output

Input Tensor:
   tensor([[[1, 2],
      [3, 4]]])
Padded Tensor:
   tensor([[[0, 0, 0, 0, 0, 0],
      [0, 0, 0, 0, 0, 0],
      [0, 0, 1, 2, 0, 0],
      [0, 0, 3, 4, 0, 0],
      [0, 0, 0, 0, 0, 0],
      [0, 0, 0, 0, 0, 0]]])
Input Tensor:
   tensor([[[[-0.8336, -0.7609, 2.2278],
      [-0.5882, -1.2273, 0.3331],
      [ 2.1541, -0.0235, -0.4785]]],

      [[[ 0.8795, 2.6868, 1.2850],
      [-1.6687, -0.8479, 0.3797],
      [-1.5313, 0.5221, -1.5769]]]])
Padded Tensor:
   tensor([[[[ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
      [ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
      [ 0.0000, 0.0000, -0.8336, -0.7609, 2.2278, 0.0000, 0.0000],
      [ 0.0000, 0.0000, -0.5882, -1.2273, 0.3331, 0.0000, 0.0000],
      [ 0.0000, 0.0000, 2.1541, -0.0235, -0.4785, 0.0000, 0.0000],
      [ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
      [ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]]],

      [[[ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
      [ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
      [ 0.0000, 0.0000, 0.8795, 2.6868, 1.2850, 0.0000, 0.0000],
      [ 0.0000, 0.0000, -1.6687, -0.8479, 0.3797, 0.0000, 0.0000],
      [ 0.0000, 0.0000, -1.5313, 0.5221, -1.5769, 0.0000, 0.0000],
      [ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
      [ 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]]]])

Example 2

In the following Python example, we pad 3D and 4D tensors using padding sizes different for all boundaries of the input tensor.

# Import the required library
import torch
import torch.nn as nn

# define 3D tensor (C,H,W)
input = torch.tensor([[[ 1, 2],[ 3, 4]]])
print("Input Tensor:
",input)

# define padding different for different sides
padding = (2,1,2,1)
pad = nn.ZeroPad2d(padding)

# pad the input tensor
output = pad(input)
print("Padded Ternsor:
", output)

input = torch.tensor([[[ 1, 2],[ 3, 4]]])
print("Input Tensor:
",input)

# define padding different for left and right sides
padding = (2,1)
pad = nn.ZeroPad2d(padding)

# pad the input tensor
output = pad(input)
print("Padded Ternsor:
", output)

# define 4D tensor (N,C,H,W)->for a batch of N tensors
input = torch.tensor([[[ 1, 2],[ 3, 4]],[[ 1, 2],[ 3, 4]]])
print("Input Tensor:
",input)

# define padding different for different sides
padding = (2,2,1,1)
pad = nn.ZeroPad2d(padding)

# pad the input tensor
output = pad(input)
print("Padded Ternsor:
", output)

Output

Input Tensor:
   tensor([[[1, 2],
      [3, 4]]])
Padded Tensor:
   tensor([[[0, 0, 0, 0, 0],
      [0, 0, 0, 0, 0],
      [0, 0, 1, 2, 0],
      [0, 0, 3, 4, 0],
      [0, 0, 0, 0, 0]]])
Input Tensor:
   tensor([[[1, 2],
      [3, 4]]])
Padded Ternsor:
   tensor([[[0, 0, 1, 2, 0],
      [0, 0, 3, 4, 0]]])
Input Tensor:
   tensor([[[1, 2],
      [3, 4]],
      [[1, 2],
      [3, 4]]])
Padded Tensor:
   tensor([[[0, 0, 0, 0, 0, 0],
      [0, 0, 1, 2, 0, 0],
      [0, 0, 3, 4, 0, 0],
      [0, 0, 0, 0, 0, 0]],

      [[0, 0, 0, 0, 0, 0],
      [0, 0, 1, 2, 0, 0],
      [0, 0, 3, 4, 0, 0],
      [0, 0, 0, 0, 0, 0]]])