Scribd
Upload
0 views45 pages

Lecture - 2 Linear Regression (Applied ML)

The document provides an overview of linear regression, a supervised statistical learning technique used for predicting quantitative responses. It discusses the process of estimating coefficients, assessing model accuracy, and applications such as sales forecasting and price estimation. Additionally, it covers the implementation of linear regression using Python, including data scaling, model fitting, and performance assessment with a focus on a Boston house prices dataset.

Uploaded by

abrham keraleme
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0 views45 pages

Lecture - 2 Linear Regression (Applied ML)

The document provides an overview of linear regression, a supervised statistical learning technique used for predicting quantitative responses. It discusses the process of estimating coefficients, assessing model accuracy, and applications such as sales forecasting and price estimation. Additionally, it covers the implementation of linear regression using Python, including data scaling, model fitting, and performance assessment with a focus on a Boston house prices dataset.

Uploaded by

abrham keraleme
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 45

Outline

● What is Linear Regression?


● How does it work?
● Estimating the coefficients
● Assessing the accuracy of the coefficient estimates
● Assessing the accuracy of the model
●
What is Linear Regression?

● Linear regression is a simple supervised statistical learning technique used


for predicting quantitative a response target.
● It is one of the simplest and oldest statistical learning techniques
● It is still very useful and widely used to this day for many reasons.
○ A good starting point for the newer and more sophisticated
machine-learning techniques, many of which may be seen as
generalizations of linear regresssion,
○ Having a good understanding of linear regression proves invaluable
in studying these newer and more sophisticated methods.
What is Regression?

● Regression is the process of predicting


a continuous value
What is Regression?

● Is that possible to predict the co2


emission of vehicle 9 before production
given engine size and cylinder size?
What is Regression?
Application of Regression

● Sales forecasting
● Satisfaction analysis
● Price estimation
● Employment Income
Linear Regression with one variable

Model Representation
Linear regression with one variable
We want to tell the price of the house given the size of the house?
Linear regression with one variable
Linear regression with one variable
Linear regression with one variable
Linear regression with one variable
Linear regression with one variable
Linear regression with one variable
Linear regression with one variable
Linear regression with one variable
Linear Regression with one variable

Gradient descent
Gradient Descent in Linear Regression?

● Any ML project our main aim relies on how good our project accuracy is or how much our
model prediction differs from the actual data point.
● Based on the difference between model prediction and actual data points we try to find the
parameters of the model which give better accuracy on our dataset
● In order to find these parameters we apply gradient descent on the cost function of the
machine learning model.
Gradient Descent in Linear Regression?
Gradient Descent in Linear Regression?
Gradient descent
Gradient descent
Gradient descent - learning rate

If we choose α to be very small, Gradient


If we choose α to be very large, Gradient Descent can
overshoot the minimum. It may fail to converge or even
Descent will take small steps to reach
diverge. local minima and will take a longer time to
reach minima.
Linear Regression with multiple variables

Multiple features
Single Variable vs multiple variables
Modeling
Modeling
Learning rate
Learning rate
Learning rate
Linear Regression with Python
Given Boston house prices dataset: Mock Scenario
● Imagine the following scenario: Your team has been contracted by a real-estate company who owns
many houses in the Boston metro-area. You are tasked to analyze if the house prices owned by the
company are competitive and if they need to be adjusted to optimize the overall profit.

Below are some questions that your team might want to address before making recommendations:

● Is there a relationship between the different predictors/features and the house prices
● If there is such a relationship, then how strong is it?
● Which features affect the most the house prices?
● How accurately can we estimate the effect of each feature to the house prices?
● How accurately can we predict the house prices based on these features?
● What kind of relationship is there between the features and the house prices? Linear?Non-linear?
● Is there any interaction effect among the features?

We will attempt to address all of these questions using linear regression.


Simple linear Regression

Let's go ahead and import some libraries:


Description of Boston house prices dataset
Description of Boston house prices dataset
Scaling data

● Many of the measurements are in different scale - It is important to scale the data before we fit a
linear regression.
Scaling data
Train-Test Split
Fitting the Model

First, we need to import Linear Regression model

Next, we instantiate and fit it to our model

Now that we have fitted our model, we can actually go ahead and extract the estimated coefficients,
including the intercept

22.605596398342957

array([-0.63082368, 0.57150473, 2.24918842, -1.09485652, 1.85711767,


-1.88803821, -2.046749 , -4.55974378])
Fitting the Model …
Assesing Performance of Model

Now that our model is fit, we need to assess the performance of our model by testing it on the test-set.

g_pred contains the predicted house prices.


Test the performance

lg_r2=r2_score(y_test,lg_pred)
mse=mean_squared_error(y_test,lg_pred)

lg_r2

mse

You might also like