Scribd
Upload
5 views7 pages

LASSO Regression

This document outlines the implementation of Lasso Regression in Python to predict employee salaries based on years of experience. It details the process of importing necessary libraries, defining the Lasso Regression class, training the model, and visualizing results using a scatter plot. The model demonstrates the relationship between experience and salary while incorporating L1 regularization for feature selection.

Uploaded by

kumar.rucky.rakesh1423
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
5 views7 pages

LASSO Regression

This document outlines the implementation of Lasso Regression in Python to predict employee salaries based on years of experience. It details the process of importing necessary libraries, defining the Lasso Regression class, training the model, and visualizing results using a scatter plot. The model demonstrates the relationship between experience and salary while incorporating L1 regularization for feature selection.

Uploaded by

kumar.rucky.rakesh1423
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 7

Implementation of Lasso Regression in

Python
It has 2 columns — “YearsExperience” and “Salary” for 30
employees in a company. So in this, we will train a Lasso
Regression model to learn the correlation between the number of
years of experience of each employee and their respective salary.
Once the model is trained, we will be able to predict the salary of
an employee on the basis of his years of experience.
Importing Libraries
Python3

# Importing libraries

import numpy as np

import pandas as pd

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import StandardScaler

import matplotlib.pyplot as plt

With a dataset, this Python method applies Lasso Regression. It


imports the required libraries, such as scikit-learn, Pandas, and
NumPy. StandardScaler is used to standardize characteristics
after the dataset is read from a CSV file. The Lasso Regression
model is then trained, the data is divided into training and testing
sets, and the outcomes are displayed using a scatter plot and the
Lasso Regression line created with Matplotlib. This code
demonstrates how to predict and choose key features in a
machine learning environment using the Lasso Regression
approach.
Defining the Lasso Regression Class
Python3

# Lasso Regression
class LassoRegression():

def __init__(self, learning_rate, iterations, l1_penalty):

self.learning_rate = learning_rate

self.iterations = iterations

self.l1_penalty = l1_penalty

# Function for model training

def fit(self, X, Y):

# no_of_training_examples, no_of_features

self.m, self.n = X.shape

# weight initialization

self.W = np.zeros(self.n)

self.b = 0

self.X = X

self.Y = Y

# gradient descent learning

for i in range(self.iterations):

self.update_weights()

return self
# Helper function to update weights in gradient descent

def update_weights(self):

Y_pred = self.predict(self.X)

# calculate gradients

dW = np.zeros(self.n)

for j in range(self.n):

if self.W[j] > 0:

dW[j] = (-2 * (self.X[:, j]).dot(self.Y - Y_pred) +

self.l1_penalty) / self.m

else:

dW[j] = (-2 * (self.X[:, j]).dot(self.Y - Y_pred) -

self.l1_penalty) / self.m

db = -2 * np.sum(self.Y - Y_pred) / self.m

# update weights

self.W = self.W - self.learning_rate * dW

self.b = self.b - self.learning_rate * db

return self

# Hypothetical function h(x)


def predict(self, X):

return X.dot(self.W) + self.b

This Python code defines a class called Lasso Regression. The


course covers initialization, training (fitting), and prediction
techniques for models. To update weights and train the model
over a predetermined number of iterations, the fit technique uses
gradient descent. The weight updates include the regularization
term (L1 penalty), which encourages sparsity in the model. Using
the learnt weights as a basis, the predict method computes the
predicted values. The basic phases of Lasso Regression are
demonstrated in this implementation, with a focus on the iterative
optimization procedure used to minimize the cost function with L1
regularization.
Training the model
Python3

def main():

# Importing dataset

df = pd.read_csv("Experience-Salary.csv")

X = df.iloc[:, :-1].values

Y = df.iloc[:, 1].values

# Standardize features

scaler = StandardScaler()

X = scaler.fit_transform(X)
# Splitting dataset into train and test set

X_train, X_test, Y_train, Y_test = train_test_split(

X, Y, test_size=1/3, random_state=0)

# Model training

model = LassoRegression(

iterations=1000, learning_rate=0.01, l1_penalty=500)

model.fit(X_train, Y_train)

# Prediction on test set

Y_pred = model.predict(X_test)

print("Predicted values: ", np.round(Y_pred[:3], 2))

print("Real values: ", Y_test[:3])

print("Trained W: ", round(model.W[0], 2))

print("Trained b: ", round(model.b, 2))

# Visualization on test set

plt.scatter(X_test, Y_test, color='blue', label='Actual Data')

plt.plot(X_test, Y_pred, color='orange', label='Lasso Regression Line')

plt.title('Salary vs Experience (Lasso Regression)')


plt.xlabel('Years of Experience (Standardized)')

plt.ylabel('Salary')

plt.legend()

plt.show()

if __name__ == "__main__":

main()

Output:
Predicted values: [19.88 44.43 34.78]
Real values: [12.40492474 42.64192391 32.61398476]
Trained W: 6.84
Trained b: 26.61
Lasso Regression

This main function shows how to use Lasso Regression on a


dataset for salary prediction. After reading the dataset and using
StandardScaler to standardize the features, it divides it into
training and testing sets. Next, the Lasso Regression model is
trained using predetermined parameters. On the test set,
predictions are made, and the model’s effectiveness is assessed
by contrasting the expected and actual values. Ultimately,
a scatter plot of the real data points and the Lasso Regression line
is used to display the results. The model’s capacity to represent
the association between years of experience and pay while using
L1 regularization for feature selection is demonstrated visually in
the visualization.
Note: It automates certain parts of model selection and
sometimes called variables eliminator.

You might also like