Introduction to deep

learning with
PyTorch
INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Maham Faisal Khan

Senior Data Science Content Developer
What is deep learning?
Deep learning is everywhere:
Language translation

Self-driving cars

Medical diagnostics

Chatbots

Being used on multiple data types: images,

text and audio

While traditional machine learning relies on

hand-crafted feature engineering, the
layered structure in deep learning enables
feature learning from raw data

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

What is deep learning?
Deep learning is a subset of machine
learning

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

What is deep learning?
Deep learning is a subset of machine
learning

Inspired by connections in the human brain

Models require large amount of data

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

PyTorch: a deep learning framework
PyTorch is
one of the most popular deep learning frameworks

the framework used in many published deep learning papers

intuitive and user-friendly

has much in common with NumPy

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Importing PyTorch and related packages
PyTorch import in Python

import torch

PyTorch supports
image data with torchvision

audio data with torchaudio

text data with torchtext

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Tensors: the building blocks of networks in PyTorch
Load from list Load from NumPy array

import torch np_array = np.array(array)

np_tensor = torch.from_numpy(np_array)
array = [[1, 2, 3], [4, 5, 6]]
tensor = torch.tensor(array) Like NumPy arrays, tensors are
multidimensional representations of their
elements

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Tensor attributes
Tensor shape Tensor device

array = [[1, 2, 3], [4, 5, 6]]

tensor.device
tensor = torch.tensor(array)
tensor.shape
device(type='cpu')

torch.Size([2, 3])
Deep learning often requires a GPU, which,
Tensor data type compared to a CPU can offer:

tensor.dtype parallel computing capabilities

faster training times

torch.int64
better performance

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Getting started with tensor operations
Compatible shapes Incompatible shapes
a = torch.tensor([[1, 1], a = torch.tensor([[1, 1],
[2, 2]]) [2, 2]])

b = torch.tensor([[2, 2], c = torch.tensor([[2, 2, 4],

[3, 3]]) [3, 3, 5]])

Addition / subtraction Addition / subtraction

a + b a + c

tensor([[3, 3], RuntimeError: The size of tensor a

[5, 5]]) (2) must match the size of tensor b (3)
at non-singleton dimension 1

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Getting started with tensor operations
Element-wise multiplication ... and much more
Transposition
a = torch.tensor([[1, 1],
[2, 2]])
Matrix multiplication

b = torch.tensor([[2, 2], Concatenation

[3, 3]])
Most NumPy array operations can be
a * b
performed on PyTorch tensors

tensor([[2, 2],
[6, 6]])

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

 Let's practice!
INTRODUCTION TO DEEP LEARNING WITH PYTORCH
 Creating our first
neural network
INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Maham Faisal Khan

Senior Data Science Content Developer
Our first neural network

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Our first neural network

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Our first neural network

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Our first neural network
import torch.nn as nn

## Create input_tensor with three features

input_tensor = torch.tensor(
[[0.3471, 0.4547, -0.2356]]
)

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Our first neural network
import torch.nn as nn

## Create input_tensor with three features

input_tensor = torch.tensor(
[[0.3471, 0.4547, -0.2356]])

A linear layer takes an input, applies a linear

function, and returns output

# Define our first linear layer

linear_layer = nn.Linear(in_features=3, out_features=2)

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Our first neural network
import torch.nn as nn

## Create input_tensor with three features

input_tensor = torch.tensor(
[[0.3471, 0.4547, -0.2356]])

# Define our first linear layer

linear_layer = nn.Linear(in_features=3, out_features=2)

# Pass input through linear layer

output = linear_layer(input_tensor)
print(output)

tensor([[-0.2415, -0.1604]],
grad_fn=<AddmmBackward0>)

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Getting to know the linear layer operation
Each linear layer has a .weight and .bias property

linear_layer.weight linear_layer.bias

Parameter containing: Parameter containing:

tensor([[-0.4799, 0.4996, 0.1123], tensor([0.0310, 0.1537],
[-0.0365, -0.1855, 0.0432]], requires_grad=True)
requires_grad=True)

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Getting to know the linear layer operation
output = linear_layer(input_tensor)

For input X , weights W0 and bias b0 , the linear layer performs

y0 = W0 ⋅ X + b0

In PyTorch: output = W0 @ input + b0

Weights and biases are initialized randomly

They are not useful until they are tuned

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Our two-layer network summary
Input dimensions: 1 × 3
Linear layer arguments:
in_features = 3

out_features = 2

Output dimensions: 1 × 2

Networks with only linear layers are called

fully connected

Each neuron in one layer is connected to

each neuron in the next layer

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Stacking layers with nn.Sequential()
# Create network with three linear layers
model = nn.Sequential(
nn.Linear(10, 18),
nn.Linear(18, 20),
nn.Linear(20, 5)
)

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Stacking layers with nn.Sequential()
print(input_tensor)

tensor([[-0.0014, 0.4038, 1.0305, 0.7521, 0.7489, -0.3968, 0.0113, -1.3844, 0.8705, -0.9743]])

# Pass input_tensor to model to obtain output

output_tensor = model(input_tensor)
print(output_tensor)

tensor([[-0.0254, -0.0673, 0.0763,

0.0008, 0.2561]], grad_fn=<AddmmBackward0>)

We obtain output of 1 × 5 dimensions

Output is still not yet meaningful

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

 Let's practice!
INTRODUCTION TO DEEP LEARNING WITH PYTORCH
 Discovering
activation functions
INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Maham Faisal Khan

Senior Data Science Content Developer
Stacked linear operations
We have only seen linear layer networks
Each linear layer multiplies its respective
input with layer weights and adds biases

Even with multiple stacked linear layers,

output still has linear relationship with input

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Why do we need activation functions?
Activation functions add non-linearity to
the network

A model can learn more complex

relationships with non-linearity

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Meet the sigmoid function
Binary classification task:

To predict whether animal is 1 (mammal) or

0 (not mammal),

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Meet the sigmoid function
Binary classification task:

To predict whether animal is 1 (mammal) or

0 (not mammal),

we take the pre-activation (6),

pass it to the sigmoid,

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Meet the sigmoid function
Binary classification task:

To predict whether animal is 1 (mammal) or

0 (not mammal),

we take the pre-activation (6),

pass it to the sigmoid,

and obtain a value between 0 and 1.

Using the common threshold of 0.5:

If output is > 0.5, class label = 1 (mammal)

If output is <= 0.5, class label = 0 (not

mammal)

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Meet the sigmoid function
import torch
import torch.nn as nn

input_tensor = torch.tensor([[6.0]])
sigmoid = nn.Sigmoid()
output = sigmoid(input_tensor)

tensor([[0.9975]])

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Activation function as the last layer
model = nn.Sequential(
nn.Linear(6, 4), # First linear layer
nn.Linear(4, 1), # Second linear layer
nn.Sigmoid() # Sigmoid activation function
)

Note. Sigmoid as last step in network of linear layers is equivalent to traditional logistic
regression.

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Getting acquainted with softmax
used for multi-class classification problems

takes N-element vector as input and

outputs vector of same size

say N=3 classes:

bird (0), mammal (1), reptile (2)

output has three elements, so softmax

has three elements

outputs a probability distribution:

each element is a probability (it's
bounded between 0 and 1)
the sum of the output vector is equal to 1

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

Getting acquainted with softmax
import torch dim = -1 indicates softmax is applied to the
import torch.nn as nn input tensor's last dimension

nn.Sigmoid() can be used as last step in

# Create an input tensor
input_tensor = torch.tensor( nn.Sequential()
[[4.3, 6.1, 2.3]])

# Apply softmax along the last dimension

probabilities = nn.Softmax(dim=-1)
output_tensor = probabilities(input_tensor)

print(output_tensor)

tensor([[0.1392, 0.8420, 0.0188]])

INTRODUCTION TO DEEP LEARNING WITH PYTORCH

 Let's practice!
INTRODUCTION TO DEEP LEARNING WITH PYTORCH