Faculty of Computing
CS-405 Deep Learning
BS-EE
Lab 2
CLO 3, CLO 4
Date: 5-02-2025
Name: Muhammad Zohaib Irfan
Lab Engineer: Ms. Areeba Rameen
Instructor: Ms. Huma Ameer
Objective
In this lab, you will:
● Implement the classic perceptron learning algorithm using PyTorch.
● Train the perceptron on a linearly separable dataset.
● Design perceptrons for an XOR dataset.
● Explore the applicability of perceptrons on a dataset of your choice.
● Answer key conceptual questions about perceptrons.
Task 1: Implement a Classic Perceptron in PyTorch
(from scratch)
Instructions
● Create a Perceptron class in PyTorch using torch.nn.Module.
● Implement the step function as the activation function (no sigmoid, just
thresholding).
● Train the perceptron using weight updates based on the Perceptron Learning
Rule:
● Do not forget the bias term!
● Train the perceptron on the AND function.
● Print weight updates after each training sample.
● After training, visualize the decision boundary to confirm learning.
Code and output:
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
class Perceptron(nn.Module):
def __init__(self, input_size):
super(Perceptron, self).__init__()
self.weights = nn.Parameter(torch.randn(input_size + 1) * 0.01)
def forward(self, x):
x = torch.cat((torch.ones(x.shape[0], 1), x), dim=1)
return torch.where(torch.matmul(x, self.weights) >= 0, 1, 0)
#and gate input data would look something like this
X = torch.tensor([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=torch.float32)
y = torch.tensor([0, 0, 0, 1], dtype=torch.float32)
# Training the perceptron
perceptron = Perceptron(input_size=2)
learning_rate = 0.1
for epoch in range(8): # We can write epochs here
total_loss = 0
for i in range(len(X)):
x_sample = X[i].unsqueeze(0)
y_sample = y[i]
prediction = perceptron(x_sample).squeeze()
error = y_sample - prediction
with torch.no_grad():
perceptron.weights += learning_rate * error *
torch.cat((torch.tensor([1.0]), x_sample.squeeze()))
print(f"Sample {X[i].tolist()}, Prediction: {prediction.item()},
Error: {error.item()}, Weights: {perceptron.weights.tolist()}")
# now to plot decision boundary
def plot_decision_boundary(perceptron, X, y):
x_min, x_max = -0.5, 1.5
y_min, y_max = -0.5, 1.5
xx, yy = np.meshgrid(np.linspace(x_min, x_max, 100), np.linspace(y_min,
y_max, 100))
grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32)
Z = perceptron(grid).reshape(xx.shape)
plt.contourf(xx, yy, Z.detach().numpy(), levels=[-1, 0.5, 1], alpha=0.3,
colors=['red', 'blue'])
plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='k', cmap=plt.cm.Paired)
plt.xlabel('X1')
plt.ylabel('X2')
plt.title('Perceptron Decision Boundary for AND Gate')
plt.show()
plot_decision_boundary(perceptron, X, y)
Task 2: Implement XOR through Perceptrons
🛠 Instructions
● Modify your perceptron code to train on the XOR dataset:
● Train the perceptron using the same Perceptron Learning Rule.
● Observe the final predictions and weight updates.
● Plot the decision boundary and analyze whether it correctly classifies XOR.
Code:
import torch
import numpy as np
import matplotlib.pyplot as plt
# Perceptron for XOR
class Perceptron:
def __init__(self, input_size, learning_rate=0.1):
self.weights = torch.randn(input_size + 1) * 0.01
self.learning_rate = learning_rate
def step_function(self, x):
return 1 if x >= 0 else 0
def predict(self, x):
x = torch.cat((torch.tensor([1.0]), x))
return self.step_function(torch.dot(self.weights, x))
def train(self, X, y, epochs=6):
for epoch in range(epochs):
for i in range(len(X)):
x_sample = X[i]
y_sample = y[i]
prediction = self.predict(x_sample)
error = y_sample - prediction
self.weights += self.learning_rate * error *
torch.cat((torch.tensor([1.0]), x_sample))
print(f"Sample {X[i].tolist()}, Prediction: {prediction},
Error: {error}, Weights: {self.weights.tolist()}")
X = torch.tensor([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=torch.float32)
y = torch.tensor([0, 1, 1, 0], dtype=torch.float32) # XOR function labels
perceptron = Perceptron(input_size=2)
perceptron.train(X, y)
def plot_decision_boundary(perceptron, X, y):
x_min, x_max = -0.5, 1.5
y_min, y_max = -0.5, 1.5
xx, yy = np.meshgrid(np.linspace(x_min, x_max, 100), np.linspace(y_min,
y_max, 100))
grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32)
Z = torch.tensor([perceptron.predict(point) for point in
grid]).reshape(xx.shape)
plt.contourf(xx, yy, Z.numpy(), levels=[-1, 0.5, 1], alpha=0.3,
colors=['red', 'blue'])
plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='k', cmap=plt.cm.Paired)
plt.xlabel('X1')
plt.ylabel('X2')
plt.title('Perceptron Decision Boundary for XOR Gate')
plt.show()
plot_decision_boundary(perceptron, X, y)
Output:
Results from different epochs:
Task 3: Apply Perceptron to a Dataset of Your Choice
Instructions
● Think of a use-case you want to solve.
● Generate a small synthetic dataset using a Large Language Model (LLM) or
choose a simple dataset that interests you.
● Check if a perceptron can solve the problem:
○ Train a single-layer perceptron on the dataset.
○ Evaluate its performance.
○ If it fails, analyze why.
● Show your working in code, including dataset generation, training, and evaluation.
● Discuss whether the problem is linearly separable and explain your findings.
Code:
Ok so I am taking a random dataset but its arranged in a circle, again it will not be a linear
data and we will see what happens, obviously a simple line wont be able to do it
import torch
import numpy as np
import matplotlib.pyplot as plt
# Perceptron Implementation for Custom Dataset
class Perceptron:
def __init__(self, input_size, learning_rate=0.1):
self.weights = torch.randn(input_size + 1) * 0.01 # Including bias
self.learning_rate = learning_rate
def step_function(self, x):
return 1 if x >= 0 else 0 # Simple threshold activation
def predict(self, x):
x = torch.cat((torch.tensor([1.0]), x)) # Add bias term
return self.step_function(torch.dot(self.weights, x))
def train(self, X, y, epochs=6):
for epoch in range(epochs):
for i in range(len(X)):
x_sample = X[i]
y_sample = y[i]
prediction = self.predict(x_sample)
error = y_sample - prediction
self.weights += self.learning_rate * error *
torch.cat((torch.tensor([1.0]), x_sample))
print(f"Sample {X[i].tolist()}, Prediction: {prediction},
Error: {error}, Weights: {self.weights.tolist()}")
def generate_circle_data(n_samples=100):
X = torch.rand((n_samples, 2)) * 2 - 1
y = torch.tensor([(1 if x[0]**2 + x[1]**2 < 0.5**2 else 0) for x in X],
dtype=torch.float32)
return X, y
X, y = generate_circle_data(100)
perceptron = Perceptron(input_size=2)
perceptron.train(X, y)
def plot_decision_boundary(perceptron, X, y):
x_min, x_max = -1.1, 1.1
y_min, y_max = -1.1, 1.1
xx, yy = np.meshgrid(np.linspace(x_min, x_max, 100), np.linspace(y_min,
y_max, 100))
grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32)
Z = torch.tensor([perceptron.predict(point) for point in
grid]).reshape(xx.shape)
plt.contourf(xx, yy, Z.numpy(), levels=[-1, 0.5, 1], alpha=0.3,
colors=['red', 'blue'])
plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='k', cmap=plt.cm.Paired)
plt.xlabel('X1')
plt.ylabel('X2')
plt.title('Perceptron Decision Boundary (Inside vs. Outside Circle)')
plt.show()
plot_decision_boundary(perceptron, X, y)
Changing epochs size:
We tried to train a single-layer perceptron to classify points as inside or outside a circle. The
perceptron learned something, but it failed to correctly separate the data. The perceptron failed
because it can only learn linear decision boundaries, but our dataset requires a nonlinear one
(a circle).
Task 4: Answer These Conceptual Questions
Write a short reflection answering these questions:
1. Does the perceptron successfully classify the AND function? Why or why not?
Yes. The AND function is linearly separable, so a perceptron can find a straight-line
decision boundary to classify it correctly.
2. Why does a single-layer perceptron fail to learn XOR?
Because XOR is not linearly separable. A single-layer perceptron can only learn straight-
line boundaries, but XOR requires a more complex separation.
3. What type of problems can a perceptron solve?
Linearly separable problems like AND, OR, and simple classification tasks where a
single straight line can split the classes
4. What modifications would allow a model to solve XOR?
Use a Multi-Layer Perceptron. Adding a hidden layer allows the model to learn complex,
nonlinear decision boundaries.
Following is output after adding multi layers and finally solving it for XOR too.
Submission Guidelines
● Your code should be written in PyTorch.
● Ensure that you print weight updates at each step.
● Include a decision boundary plot for both AND and XOR.
● Submit a brief explanation answering the conceptual questions.
Hints
● Remember to add a bias term to your input features!
● Use a step function (thresholding) for activation, not sigmoid.
● The perceptron should update weights after every training sample, not after the
entire dataset.
● Try visualizing the decision boundary to understand model behavior.
Important Note:
● Copied labs will be marked zero.
○ All submissions must reflect your own understanding and effort.
Deadline:
● Submission Deadline for this task is 11:00 am Thursday-(6-Feb-2025)
If you have any questions or need assistance, please reach out to the lab engineer.
