Scribd
Upload
17 views11 pages

Linear Regression

Uploaded by

dishu.diwanshi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
17 views11 pages

Linear Regression

Uploaded by

dishu.diwanshi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 11

Linear Regression

Introduction
Why should we learn Linear Regression
&
How are companies using it??
Why to Learn this Model?
• Provides foundation to predictive modelling.
• How to fit a model on the data?
• How predictive modelling works?
• Helps in creating a foundation for advanced concepts of Deep Learning or
Neural Network
• Easy to understand and interpret
• Served as a benchmark and its performance can be compared againt
advanced algorithms.
• Creates a basic foundation for Loss function, regularization and Gradient
Descent.
Basics of Linear Regression
• Models relationship between dependent variable Y, and one or more
independent X variables.
Equation for that would be
Y=β0​+β1​X+ϵ. (Simple Linear Regression)
Y=β0​+β1​X1​+β2​X2​+...+βn​Xn​+ϵ.(Multiple Linear Regression)
• What are these β0 and β1?
β this is the coefficient
• Intercept (0β0​): The expected mean value of Y when all X are 0.
• Example: In a simple linear regression model like Y=β0​+β1​X, if β0​is 5,
it means when X is 0, the expected value of Y is 5.
• Slope Coefficients (β1​,β2​,...,βn​):Slope coefficients represent the
expected change in the dependent variable (Y) for a one-unit change in
an independent variable, assuming all other variables remain constant.
• Multiple Variables: In multiple linear regression, each independent
variable Xi​has its own slope coefficient βi​. This coefficient tells you
how much Y changes for a unit change in Xi​, holding other variables
constant
• Positive and Negative Slopes:
• A positive slope (βi​>0) indicates that as Xi​increases, Y also increases.
• A negative slope (βi​<0) indicates that as Xi​increases, Y decreases.
• Example: In a model like β0​+β1​X1​+β2​X2​, if β1​is 3, it suggests that for
each one-unit increase in X1​, Y increases by 3 units, assuming X2​
remains constant.
• It's crucial to interpret these coefficients in the context of the data. For
example, in a regression model predicting house prices (Y), the slope
coefficient for the size of the house (X1​) might tell you how much the
price is expected to increase for each additional square foot.
• Error Term (ϵ):Represents the difference between the observed
values and the predicted values. It accounts for the variability in Y that
cannot be explained by X.
Assumptions of Linear Regression

• Linearity: The relationship between X


and Y is linear.
• Independence: Observations are
independent of each other.
• Homoscedasticity: The variance of
residual is the same for any value of X.
• Normal Distribution of Residuals:
Residuals (errors) are normally
distributed.
How can we estimate the coefficients and
how to minimize the residuals?
• Least Squares Method:
• A method used to estimate the coefficients. It minimizes the sum of the
squares of the residuals, ensuring the best fit line.
How to determine the best fit line?
1. Residuals:
• A residual is the difference between an observed value and the value predicted by the model.
• In a formula, if yi​is the observed value and y^​i​is the predicted value for the ith observation, the residual ei​is ei​=yi​−y^​
i​.
2. Objective of Least Squares:
• The goal is to find the line that minimizes the sum of the squares of these residuals. This is why it's called 'least
squares'.
• The sum of squared residuals (SSR) is ∑i=1n​ei2​, where n is the number of observations
3. Minimization:
• Mathematically, this involves partial derivatives. We differentiate the SSR
with respect to each coefficient (intercept and slopes), set the derivatives equal
to zero, and solve the resulting system of equations.
• This process yields the estimates of the coefficients that minimize the SSR.
• Our goal is to find the estimates of β0​and β1​that minimize the sum of
squared residuals (SSR). The SSR is given by:

You might also like