Scribd
Upload
28 views

Pytorch Tutorial 2

This document provides an overview of PyTorch documentation and common errors. It includes: 1. An introduction to key PyTorch documentation pages for neural networks (torch.nn), optimization algorithms (torch.optim), and datasets (torch.utils.data). 2. Examples of PyTorch documentation usage, including different argument types like positional vs. keyword arguments and default values. 3. Examples of common errors like tensors on different devices than the model, mismatched tensor dimensions, out of memory errors, and mismatched tensor types. Troubleshooting steps are provided for each error.

Uploaded by

zhangchelsea9
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
28 views

Pytorch Tutorial 2

This document provides an overview of PyTorch documentation and common errors. It includes: 1. An introduction to key PyTorch documentation pages for neural networks (torch.nn), optimization algorithms (torch.optim), and datasets (torch.utils.data). 2. Examples of PyTorch documentation usage, including different argument types like positional vs. keyword arguments and default values. 3. Examples of common errors like tensors on different devices than the model, mismatched tensor dimensions, out of memory errors, and mismatched tensor types. Troubleshooting steps are provided for each error.

Uploaded by

zhangchelsea9
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

Machine Learning

Pytorch Tutorial 2
Documentation and Common Errors
TA: 石旻翰
2023.02.20
[email protected]
PyTorch Documentation
https://pytorch.org/docs/stable/

● torch.nn -> Neural Network

● torch.optim -> Optimization Algorithms

● torch.utils.data -> Dataset, Dataloader


PyTorch Documentation Example

Function inputs and outputs

Data type and explanation of each input


PyTorch Documentation Example
● Some functions behave
differently with different inputs
● Parameters : You don’t need to
specify the name of the
argument (Positional Arguments)
● Keyword Arguments : You have
to specify the name of the
argument

They are separated by *


PyTorch Documentation Example

● Some functions behave


differently with different inputs

● Arguments with default value :


Some arguments have a default
value (keepdim=False), so
passing a value of this argument
is optional
PyTorch Documentation Example
Three Kinds of torch.max
1. torch.max(input) → Tensor
2. torch.max(input, dim, keepdim=False, *,
out=None) → (Tensor, LongTensor)
3. torch.max(input, other, *, out=None) →
Tensor
input : Tensor, dim : int, keepdim : bool
other : Tensor
PyTorch Documentation Example
1.torch.max(input) → Tensor
Find the maximum value of a tensor, and return that value.
PyTorch Documentation Example
2. torch.max(input, dim, keepdim=False, *,
out=None) → (Tensor, LongTensor)
Find the maximum
value of a tensor
along a dimension,
and return that value,
along with the index
corresponding to that
value.
PyTorch Documentation Example
3.torch.max(input, other) → Tensor

Perform element-wise
comparison between two
tensors of the same size,
and select the maximum of
the two to construct a
tensor with the same size.
Common Errors - torch.max (Colab)
Three Kinds of torch.max Colab code
1. torch.max(input) x = torch.randn(4,5)
→ Tensor y = torch.randn(4,5)
2. torch.max(input, m, idx = torch.max(x,0,False,p)→x
dim, keepdim=False,
*out is a keyword argument
*, out=None) →
m, idx = torch.max(x,True)→x
(Tensor, LongTensor)
3. torch.max(input, *did not specify dim
other,
*, out=None) → Tensor
input : Tensor
dim : int
keepdim : bool
other : Tensor
Common Errors - Tensor on Different Device to Model
model = torch.nn.Linear(5,1).to("cuda:0")
x = torch.randn(5).to("cpu")
y = model(x)
Tensor for * is on CPU, but expected them to be on GPU
=> send the tensor to GPU
x = torch.randn(5).to("cuda:0")
y = model(x)
print(y.shape)
Common Errors - Mismatched Dimensions
x = torch.randn(4,5)
y = torch.randn(5,4)
z = x + y
The size of tensor a (5) must match the size of tensor b (4) at non-singleton
dimension 1

=> the shape of a tensor is incorrect, use transpose, squeeze, unsqueeze to align
the dimensions
y = y.transpose(0,1)
z = x + y
print(z.shape)
Common Errors - Cuda Out of Memory
import torch
import torchvision.models as models
resnet18 = models.resnet18().to( "cuda:0" ) # Neural Networks for Image
Recognition
data = torch.randn( 512,3,244,244) # Create fake data (512
images)
out = resnet18(data.to( "cuda:0" )) # Use Data as Input and Feed to
Model
print(out.shape)
CUDA out of memory. Tried to allocate 350.00 MiB (GPU 0; 14.76 GiB total
capacity; 11.94 GiB already allocated; 123.75 MiB free; 13.71 GiB reserved in
total by PyTorch)

=> The batch size of data is too large to fit in the GPU. Reduce the batch size.
Common Errors - Mismatched Tensor Type
import torch.nn as nn
L = nn.CrossEntropyLoss()
outs = torch.randn(5,5)
labels = torch.Tensor([1,2,3,4,0])
lossval = L(outs, labels) # Calculate CrossEntropyLoss between outs and labels

expected scalar type Long but found Float


=> labels must be long tensors, cast it to type “Long” to fix this issue
labels = labels.long()
lossval = L(outs,labels)
print(lossval)
Any Question?

You might also like