Scribd
Upload
33 views4 pages

2-Notes - Linear Regression-1

Linear regression models the relationship between independent variables (X) and a dependent variable (y) by fitting a linear equation to observed data. The goal is to find the weights that minimize a loss function, typically mean squared error, between the actual and predicted y values. Model performance is measured by the R-squared metric, with a value closer to 1 indicating a better fit to the data.

Uploaded by

saidaraovenkat62
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
33 views4 pages

2-Notes - Linear Regression-1

Linear regression models the relationship between independent variables (X) and a dependent variable (y) by fitting a linear equation to observed data. The goal is to find the weights that minimize a loss function, typically mean squared error, between the actual and predicted y values. Model performance is measured by the R-squared metric, with a value closer to 1 indicating a better fit to the data.

Uploaded by

saidaraovenkat62
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Linear Regression - 1

How does the data look for Linear Regression?


(𝑛 ×𝑑)
Regression Problem → Data: n samples with [𝑓1, 𝑓2,....., 𝑓𝑑]ϵ 𝑅 where
- sample 𝑋𝑖 consists the features
- That has a target label 𝑦𝑖 ϵ 𝑅 .

How does the training data look for the Regression problem?

Regression → Supervised task, the target 𝑦𝑖 is numerical


𝑛𝑋𝑑 𝑇
1. Input sample = 𝑋𝑖 ϵ 𝑅 , 𝑥𝑖 = [𝑥𝑖1,....., 𝑥𝑖𝑑]
2. Output sample = 𝑦𝑖 ϵ 𝑅
3. Training example = (𝑋𝑖,𝑦𝑖)
4. Training Dataset = {(𝑋𝑖,𝑦𝑖), 𝑖 = 1, 2,.. 𝑛} ,

What will be the simplest model for predicting a value?

Ans: Mean model → the mean of the entire data as its prediction.
What is the goal of the ML model?

Ans: To find 𝑓: 𝑋 → 𝑦 such that 𝑓(𝑥𝑖) ≈ 𝑦𝑖

How do we define function f?


^
Algebraic Intuition → find 𝑦 = 𝑓(𝑥𝑖),

we can say for Linear Regression:


𝑓(𝑥𝑖1, 𝑥𝑖2, ..... , 𝑥𝑖𝑑) = 𝑤1𝑥𝑖1 + 𝑤2𝑥𝑖2 + 𝑤3𝑥𝑖3 +.... + 𝑤𝑑𝑥𝑖𝑑 + 𝑤0
𝐷
^
𝑦𝑖 = 𝑓( 𝑥𝑖) = ∑ 𝑤𝑗𝑥𝑖𝑗 + 𝑤0
𝑗=1

𝑇
Now, 𝑤 = [𝑤1, 𝑤2..... 𝑤𝑑] and 𝑥𝑖 = [𝑥𝑖1, 𝑥𝑖2..... 𝑥𝑖𝑑] , then:
^ 𝑇
𝑦𝑖 = 𝑓(𝑥𝑖) = 𝑤 𝑥𝑖 + 𝑤0

How does the ML model find the function f?

Ans: By updating the weights of the model on the training dataset

Is 𝑓(𝑥𝑖) in Linear Regression analogous to 𝑦 = 𝑚𝑥 + 𝑐 ?

Ans: Yes, it is.

Linear Regression: finding the best D Dimensional hyperplane that fits the
^
D-dimensional data such that 𝑦𝑞 ≈ 𝑦𝑞

How to find the best-fit line of the Linear Regression model?

𝑇
Ans: By optimizing the weights vector 𝑊 = [𝑤1, 𝑤2,.....,, 𝑤𝑑] w.r.t the loss
function.
How do you say Linear Regression is optimized?

Ans: when we see the loss function is not decreasing anymore i.e. local minima

Sklearn-code

What loss function to use for linear regression optimization?


^
Mean Square Error→finds the mean of the square difference between 𝑦 , 𝑦 .
𝑛
1 ^ 2
𝑚𝑖𝑛𝑤,𝑤 𝑛
∑ (𝑦𝑖 − 𝑦𝑖)
0 𝑖=1

After training the model, how do we measure model performance?

Ans: R-squared metric. → measures the performance of Linear Regression


over a mean model. It is Defined as:
𝑛
^ 2
𝑆𝑆𝑟𝑒𝑠 ∑ (𝑦𝑖−𝑦𝑖)
2 −
𝑅 = 1 − 𝑆𝑆𝑡𝑜𝑡𝑎𝑙
= 1− 𝑖=1
𝑛 , where 𝑦 𝑖 is the mean model
− 2
∑ (𝑦𝑖−𝑦𝑖 )
𝑖=1

SSres - Squared sum of error of regression line


SStotal-(total sum of squares) Squared sum of error of mean line

Range: Practically, R2 ranges from (0, 1]

Sklearn code
2
What will be the best value of 𝑅 ?
𝑛
^ 2
Ans: 1, when ∑ (𝑦𝑖 − 𝑦𝑖) = 0.
𝑖=1

2
What will be the minimum value of 𝑅 ?
𝑛 𝑛
^ 2 − 2
Ans: - ∞ , when ∑ (𝑦𝑖 − 𝑦𝑖) >> ∑ (𝑦𝑖 − 𝑦𝑖 )
𝑖=1 𝑖=1

You might also like