Scribd
Upload
41 views20 pages

Lecture 06 Logistic Regression

The document discusses logistic regression in PyTorch. It begins by reviewing linear regression and classification problems. It then introduces the logistic function to map model outputs between 0 and 1 for classification. Logistic regression replaces the linear unit in linear regression with a logistic regression unit. The loss function for binary classification is also introduced. Finally, the implementation of logistic regression in PyTorch is demonstrated, including defining the model as a class, calculating loss, and using an optimizer for training.

Uploaded by

lingyun wu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
41 views20 pages

Lecture 06 Logistic Regression

The document discusses logistic regression in PyTorch. It begins by reviewing linear regression and classification problems. It then introduces the logistic function to map model outputs between 0 and 1 for classification. Logistic regression replaces the linear unit in linear regression with a logistic regression unit. The loss function for binary classification is also introduced. Finally, the implementation of logistic regression in PyTorch is demonstrated, including defining the model as a class, calculating loss, and using an optimizer for training.

Uploaded by

lingyun wu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

PyTorch Tutorial

06. Logistic Regression

Lecturer : Hongpu Liu Lecture 6-1 PyTorch Tutorial @ SLAM Research Group
Revision - Linear Regression

𝑦
Linear Unit

𝑥 ∗ + 𝑦ො 𝑙𝑜𝑠𝑠(𝑦,
ො 𝑦) 𝑙𝑜𝑠𝑠

𝜔 𝑏

Affine Model Loss Function

𝑦ො = 𝑥 ∗ 𝜔 + 𝑏 𝑙𝑜𝑠𝑠 = (𝑦ො − 𝑦)2 = (𝑥 ∙ 𝜔 − 𝑦)2

Lecturer : Hongpu Liu Lecture 6-2 PyTorch Tutorial @ SLAM Research Group
Revision - Linear Regression

x (hours) y (points)
1 2
2 4
3 6
4 ?

Affine Model Loss Function

𝑦ො = 𝑥 ∗ 𝜔 + 𝑏 𝑙𝑜𝑠𝑠 = (𝑦ො − 𝑦)2 = (𝑥 ∙ 𝜔 − 𝑦)2

Lecturer : Hongpu Liu Lecture 6-3 PyTorch Tutorial @ SLAM Research Group
Classification – The MNIST Dataset

The database of handwritten digits


- Training set: 60,000 examples,
- Test set: 10,000 examples.
- Classes: 10

import torchvision
train_set = torchvision.datasets.MNIST(root='../dataset/mnist', train=True, download=True)
test_set = torchvision.datasets.MNIST(root='../dataset/mnist', train=False, download=True)

Lecturer : Hongpu Liu Lecture 6-4 PyTorch Tutorial @ SLAM Research Group
Classification – The CIFAR-10 dataset

- Training set: 50,000 examples,


- Test set: 10,000 examples.
- Classes: 10

import torchvision
train_set = torchvision.datasets.CIFAR10(…)
test_set = torchvision.datasets.CIFAR10(…)

Lecturer : Hongpu Liu Lecture 6-5 PyTorch Tutorial @ SLAM Research Group
Regression vs Classification

x (hours) y (points) x (hours) y (pass/fail)


1 2 1 0 (fail)
2 4 2 0 (fail)
3 6 3 1 (pass)
4 ? 4 ?

In classification, the output of model is the


probability of input belongs to the exact class.

Lecturer : Hongpu Liu Lecture 6-6 PyTorch Tutorial @ SLAM Research Group
How to map: ℝ → 0, 1

Logistic Function

1
𝜎(𝑥) =
1 + 𝑒 −𝑥

https://en.wikipedia.org/wiki/Logistic_function

Lecturer : Hongpu Liu Lecture 6-7 PyTorch Tutorial @ SLAM Research Group
Sigmoid functions

Lecturer : Hongpu Liu Lecture 6-8 PyTorch Tutorial @ SLAM Research Group
Logistic Regression Model

Affine Model Logistic Regression Model

𝑦ො = 𝑥 ∗ 𝜔 + 𝑏 𝑦ො = 𝜎(𝑥 ∗ 𝜔 + 𝑏)

Linear Unit Logistic Regression Unit

𝑥 ∗ + 𝑦ො 𝑥 ∗ + 𝜎 𝑦ො

𝜔 𝑏 𝜔 𝑏

Lecturer : Hongpu Liu Lecture 6-9 PyTorch Tutorial @ SLAM Research Group
Loss function for Binary Classification

Loss Function for Linear Regression

𝑙𝑜𝑠𝑠 = (𝑦ො − 𝑦)2 = (𝑥 ∙ 𝜔 − 𝑦)2

Loss Function for Binary Classification

𝑙𝑜𝑠𝑠 = −(𝑦 log 𝑦ො + (1 − 𝑦) log(1 − 𝑦))


ො

Lecturer : Hongpu Liu Lecture 6-10 PyTorch Tutorial @ SLAM Research Group
Mini-Batch Loss function for Binary Classification

Loss Function for Binary Classification 𝒚 ෝ


𝒚 BCE Loss

𝑙𝑜𝑠𝑠 = −(𝑦 log 𝑦ො + (1 − 𝑦) log(1 − 𝑦))


ො 1 0.2 1.6094

1 0.8 0.2231

0 0.3 0.3567
Mini-Batch Loss Function for Binary Classification
𝑁 0 0.7 1.2040
1
𝑙𝑜𝑠𝑠 = − ෍ 𝑦𝑛 log 𝑦ො𝑛 + (1 − 𝑦𝑛 ) log(1 − 𝑦ො𝑛 ) Mini-Batch Loss 0.8483
𝑁
𝑛=1

Lecturer : Hongpu Liu Lecture 6-11 PyTorch Tutorial @ SLAM Research Group
Implementation of Logistic Regression
Linear Unit Logistic Regression Unit

𝑥 ∗ + 𝑦ො 𝑥 ∗ + 𝜎 𝑦ො

𝜔 𝑏 𝜔 𝑏

import torch.nn.functional as F

class LinearModel(torch.nn.Module): class LogisticRegressionModel(torch.nn.Module):


def __init__(self): def __init__(self):
super(LinearModel, self).__init__() super(LogisticRegressionModel, self).__init__()
self.linear = torch.nn.Linear(1, 1) self.linear = torch.nn.Linear(1, 1)

def forward(self, x): def forward(self, x):


y_pred = self.linear(x) y_pred = F.sigmoid(self.linear(x))
return y_pred return y_pred

Lecturer : Hongpu Liu Lecture 6-12 PyTorch Tutorial @ SLAM Research Group
Implementation of Logistic Regression

Mini-Batch Loss Function for Binary Classification


𝑁
1
𝑙𝑜𝑠𝑠 = − ෍ 𝑦𝑛 log 𝑦ො𝑛 + (1 − 𝑦𝑛 ) log(1 − 𝑦ො𝑛 )
𝑁
𝑛=1

criterion = torch.nn.BCELoss(size_average=False)

Lecturer : Hongpu Liu Lecture 6-13 PyTorch Tutorial @ SLAM Research Group
Implementation of Logistic Regression

x_data = torch.Tensor([[1.0], [2.0], [3.0]])


Prepare dataset
1
y_data = torch.Tensor([[0], [0], [1]])
#-------------------------------------------------------#
class LogisticRegressionModel(torch.nn.Module): we shall talk about this later
def __init__(self):
super(LogisticRegressionModel, self).__init__()
self.linear = torch.nn.Linear(1, 1)

def forward(self, x):


y_pred = F.sigmoid(self.linear(x))
return y_pred
model = LogisticRegressionModel()
#-------------------------------------------------------#
criterion = torch.nn.BCELoss(size_average=False)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
#-------------------------------------------------------#
for epoch in range(1000):
y_pred = model(x_data)
loss = criterion(y_pred, y_data)
print(epoch, loss.item())

optimizer.zero_grad()
loss.backward()
optimizer.step()

Lecturer : Hongpu Liu Lecture 6-14 PyTorch Tutorial @ SLAM Research Group
Implementation of Logistic Regression

x_data = torch.Tensor([[1.0], [2.0], [3.0]])


y_data = torch.Tensor([[0], [0], [1]])
#-------------------------------------------------------#
class LogisticRegressionModel(torch.nn.Module):
def __init__(self):
super(LogisticRegressionModel, self).__init__()
self.linear = torch.nn.Linear(1, 1)
Design model using Class
def forward(self, x):
y_pred = F.sigmoid(self.linear(x))
2
inherit from nn.Module
return y_pred
model = LogisticRegressionModel()
#-------------------------------------------------------#
criterion = torch.nn.BCELoss(size_average=False)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
#-------------------------------------------------------#
for epoch in range(1000):
y_pred = model(x_data)
loss = criterion(y_pred, y_data)
print(epoch, loss.item())

optimizer.zero_grad()
loss.backward()
optimizer.step()

Lecturer : Hongpu Liu Lecture 6-15 PyTorch Tutorial @ SLAM Research Group
Implementation of Logistic Regression

x_data = torch.Tensor([[1.0], [2.0], [3.0]])


y_data = torch.Tensor([[0], [0], [1]])
#-------------------------------------------------------#
class LogisticRegressionModel(torch.nn.Module):
def __init__(self):
super(LogisticRegressionModel, self).__init__()
self.linear = torch.nn.Linear(1, 1)

def forward(self, x):


y_pred = F.sigmoid(self.linear(x))
return y_pred
model = LogisticRegressionModel()
#-------------------------------------------------------#
criterion = torch.nn.BCELoss(size_average=False)
Construct loss and optimizer
3
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
#-------------------------------------------------------#
for epoch in range(1000): using PyTorch API
y_pred = model(x_data)
loss = criterion(y_pred, y_data)
print(epoch, loss.item())

optimizer.zero_grad()
loss.backward()
optimizer.step()

Lecturer : Hongpu Liu Lecture 6-16 PyTorch Tutorial @ SLAM Research Group
Implementation of Logistic Regression

x_data = torch.Tensor([[1.0], [2.0], [3.0]])


y_data = torch.Tensor([[0], [0], [1]])
#-------------------------------------------------------#
class LogisticRegressionModel(torch.nn.Module):
def __init__(self):
super(LogisticRegressionModel, self).__init__()
self.linear = torch.nn.Linear(1, 1)

def forward(self, x):


y_pred = F.sigmoid(self.linear(x))
return y_pred
model = LogisticRegressionModel()
#-------------------------------------------------------#
criterion = torch.nn.BCELoss(size_average=False)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
#-------------------------------------------------------#
for epoch in range(1000):
y_pred = model(x_data)
loss = criterion(y_pred, y_data)
print(epoch, loss.item())
Training cycle
optimizer.zero_grad() 4
loss.backward() forward, backward, update
optimizer.step()

Lecturer : Hongpu Liu Lecture 6-17 PyTorch Tutorial @ SLAM Research Group
Implementation of Logistic Regression

x_data = torch.Tensor([[1.0], [2.0], [3.0]])


Prepare dataset
1
y_data = torch.Tensor([[0], [0], [1]])
#-------------------------------------------------------#
class LogisticRegressionModel(torch.nn.Module): we shall talk about this later
def __init__(self):
super(LogisticRegressionModel, self).__init__()
self.linear = torch.nn.Linear(1, 1)
Design model using Class
def forward(self, x): 2
y_pred = F.sigmoid(self.linear(x)) inherit from nn.Module
return y_pred
model = LogisticRegressionModel()
#-------------------------------------------------------#
criterion = torch.nn.BCELoss(size_average=False)
Construct loss and optimizer
3
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
#-------------------------------------------------------#
for epoch in range(1000): using PyTorch API
y_pred = model(x_data)
loss = criterion(y_pred, y_data)
print(epoch, loss.item())
Training cycle
optimizer.zero_grad() 4
loss.backward() forward, backward, update
optimizer.step()

Lecturer : Hongpu Liu Lecture 6-18 PyTorch Tutorial @ SLAM Research Group
Result of Logistic Regression

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(0, 10, 200)


x_t = torch.Tensor(x).view((200, 1))
y_t = model(x_t)
y = y_t.data.numpy()
plt.plot(x, y)
plt.plot([0, 10], [0.5, 0.5], c='r')
plt.xlabel('Hours')
plt.ylabel('Probability of Pass')
plt.grid()
plt.show()

Lecturer : Hongpu Liu Lecture 6-19 PyTorch Tutorial @ SLAM Research Group
PyTorch Tutorial
06. Logistic Regression

Lecturer : Hongpu Liu Lecture 6-20 PyTorch Tutorial @ SLAM Research Group

You might also like