Scribd
Upload
12 views

03 Regression

The document provides an overview of regression analysis within the context of machine learning, highlighting its role in predicting quantities based on relationships between dependent and independent variables. It discusses various regression algorithms, particularly linear regression, and emphasizes the importance of evaluating model performance through error metrics such as Mean Absolute Error and Mean Squared Error. Applications of regression include predicting house prices, student grades, and crop yields.

Uploaded by

hrhee1atl
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
12 views

03 Regression

The document provides an overview of regression analysis within the context of machine learning, highlighting its role in predicting quantities based on relationships between dependent and independent variables. It discusses various regression algorithms, particularly linear regression, and emphasizes the importance of evaluating model performance through error metrics such as Mean Absolute Error and Mean Squared Error. Applications of regression include predicting house prices, student grades, and crop yields.

Uploaded by

hrhee1atl
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 39

Regression

Course: Artificial Intelligence


Fundamentals

Instructor: Marco Bonzanini


Machine Learning Tasks

Supervised Unsupervised
Discrete Data

Classification Clustering
(predict a label) (group similar items)

Continuous Data
Dimensionality
Regression Reduction
(predict a quantity) (reduce n. of variables)
Machine Learning Tasks

Supervised Unsupervised
Discrete Data

Classification Clustering
(predict a label) (group similar items)

Continuous Data
Dimensionality
Regression Reduction
(predict a quantity) (reduce n. of variables)
Classification vs Regression

• Classification:
- supervised
- requires a set of labelled training samples
- predicts a label

• Regression:
- supervised
- requires a set of labelled training samples
- predicts a quantity
Section Agenda

• Introduction to Regression Analysis

• Regression Algorithms

• Linear Regression

• Evaluation
Introduction to
Regression
Regression Analysis
• Set of process for estimating the relationship
between variables

• Heavily studied in statistics

• Can be used to better understand relationships


between variables, although “correlation does not
imply causation”

• Relationship with ML: predictions / forecasting


Applications

• Predict the price of a house

• Predict student’s grades


(time on books vs time on Facebook)

• Predict crop yields

• … Predict a quantity
Variables
• Dependent variables:
- variables that we want to forecast
- their values depend on something else
- denoted as Y (desired output)

• Independent variables:
- variables that explain the other one
- its values are independent
- denoted as X (input)
- a.k.a. features, predictors, covariates
Regression Model

• A function that describes the relationship between


dependent and independent variables

• b: unknown parameters
Regression Example
Regression Example
Price

Problem:
predict the price of a house
given its size
Size
Regression Example
Price

Training data:
samples (size, price)

Size
Regression Example
Price

Training data:
samples (size, price)
$200K

Size
100 sq. m
Regression Example
Price

Training:
find the curve
that best fits the data

Size
Regression Example
Price

Prediction:
given the size of a new house

Size
120 sq. m
Regression Example
Price

$250K
Prediction:
given the size of a new house
estimate the price

Size
120 sq. m
Regression Example
Price

$250K Estimated
$220K Real

Size
Regression Example
Price

$250K

$220K Error:
difference between
estimated and real

Size
Regression
Algorithms
Linear Regression
Linear Regression
• From school (calculus?):

• In statistics:
Linear Regression
• From school (calculus?):

• In statistics: Independent variable


Linear Regression
• From school (calculus?):

• In statistics: Dependent variable


Linear Regression
• From school (calculus?):

• In statistics: Slope
Linear Regression
• From school (calculus?):

• In statistics: Intersect
Linear Regression Error

Price

Actual value
Error:
difference between
estimated and actual
Estimated value

Size
Linear Regression Error

Price

Error:
difference between
estimated and actual

Size
Linear Regression Error (2)
Linear Regression Error (2)

Actual value

Estimated value
Error a.k.a. residual
Linear Regression
Ingredients
• Training data

• Residual error of sample i

• Sum of Squared Residuals


Linear Regression
Approach
• Minimise the SSR function

• a.k.a. Least-Squares Regression

• Treat SSR as “cost function”

• e.g. run Gradient Descent (forward reference)


Evaluation
Regression Evaluation

• Focus on the “error” (smaller = better)

• Mean Absolute Error (MAE)

• Mean Squared Error (MSE)

• Root Mean Squared Error (RMSE)


Mean Absolute Error
Mean Squared Error
Root Mean Squared Error
MAE vs MSE vs RMSE

• One better than the others?

• Squared: penalise bigger errors more

• RMSE: same unit as the variable being considered


Questions?

You might also like