Scribd
Upload
11 views4 pages

Linear Regression Notes

Linear regression is a fundamental supervised learning algorithm used for predicting continuous target variables based on input features by fitting a linear equation to observed data. It can be simple with one input variable or multiple with several features, and it aims to minimize the difference between predicted and actual values using techniques like gradient descent. While it has advantages such as ease of implementation and speed, it also has limitations, including poor performance with non-linear relationships and sensitivity to outliers.

Uploaded by

sam.music.mail
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
11 views4 pages

Linear Regression Notes

Linear regression is a fundamental supervised learning algorithm used for predicting continuous target variables based on input features by fitting a linear equation to observed data. It can be simple with one input variable or multiple with several features, and it aims to minimize the difference between predicted and actual values using techniques like gradient descent. While it has advantages such as ease of implementation and speed, it also has limitations, including poor performance with non-linear relationships and sensitivity to outliers.

Uploaded by

sam.music.mail
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
You are on page 1/ 4

Linear Regression in Machine Learning

Linear regression is one of the most fundamental and widely used algorithms in
machine learning. It is a supervised learning technique used for predicting a
continuous target variable based on one or more input features.

What is Linear Regression?


At its core, linear regression models the relationship between input variables
(independent variables) and an output variable (dependent variable) by fitting a
linear equation to observed data. The simplest form, simple linear regression,
involves a single input variable and is represented by the equation:

𝑚
=

𝑏
+

y=mx+b
y is the predicted value (output)

x is the input feature (independent variable)

m is the slope of the line (coefficient)

b is the y-intercept (bias)

For multiple linear regression, where there are multiple input features, the model
generalizes to:

𝑤
=

𝑥
1

𝑤
+

𝑥
2

2
+
⋯

𝑤
+

𝑛
𝑥
𝑛

𝑏
+

y=w
1

x
1

+w
2

x
2

+⋯+w
n

x
n

+b

𝑤
Here,

𝑤
,

2
,
…

𝑤
,

𝑛
w
1

,w
2

,…,w
n

are the weights (coefficients) associated with each feature.

How It Works
The goal of linear regression is to find the best-fitting line through the data
that minimizes the difference between the predicted values and the actual values.
This difference is measured using a loss function, typically the Mean Squared Error
(MSE):

MSE
=

𝑛
1

𝑖
∑

𝑛
1

𝑦
(

𝑦
−

𝑖
^

)
2
MSE=
n
1
i=1
∑
n

(y
i

−
y
^

)
2

where:

𝑦
𝑖
y
i

is the actual value

𝑖
^

y
^

is the predicted value

𝑛
n is the number of data points

To minimize this error, optimization techniques like gradient descent or direct


methods like the normal equation are used.

Assumptions of Linear Regression


Linear regression relies on several assumptions:

Linearity: The relationship between the inputs and output is linear.

Independence: Observations are independent of each other.

Homoscedasticity: Constant variance of errors.

Normality: Residuals (errors) are normally distributed.

Violating these assumptions can reduce the effectiveness of the model.

Applications
Linear regression is used in various domains including:
Economics (predicting costs, revenues)

Healthcare (estimating disease progression)

Marketing (forecasting sales)

Education (predicting student performance)

Advantages and Disadvantages


Advantages
Easy to implement and interpret

Fast to train, even on large datasets

Works well with linearly separable data

Disadvantages
Poor performance with non-linear relationships

Sensitive to outliers

Assumes linearity and independence among predictors

Conclusion
Linear regression serves as a foundational technique in machine learning and
statistics. While it may not always produce the most powerful predictions in
complex scenarios, it offers clarity, efficiency, and a solid baseline for
regression tasks. Mastering linear regression is a crucial step toward
understanding more advanced machine learning models.

You might also like