Scribd
Upload
32 views10 pages

Lecture 3

Linear regression is a supervised machine learning algorithm that finds the relationship between variables to predict a target value. It works by finding the best fit linear equation to model the relationship between an input feature (x) and output value (y). The model is optimized by minimizing a loss function called the residual sum of squares to calculate the optimal values for the parameters (thetas) that best represent the linear relationship in the data. The learning process involves initializing the thetas randomly and iteratively adjusting them through gradient descent by taking small steps in the direction that minimizes the loss until convergence is reached.

Uploaded by

salemamr1010
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
32 views10 pages

Lecture 3

Linear regression is a supervised machine learning algorithm that finds the relationship between variables to predict a target value. It works by finding the best fit linear equation to model the relationship between an input feature (x) and output value (y). The model is optimized by minimizing a loss function called the residual sum of squares to calculate the optimal values for the parameters (thetas) that best represent the linear relationship in the data. The learning process involves initializing the thetas randomly and iteratively adjusting them through gradient descent by taking small steps in the direction that minimizes the loss until convergence is reached.

Uploaded by

salemamr1010
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 10

Zewail City UST 2023

Introduction to Machine Learning


ENG: Ibrahim Baza

A- Linear Regression
A- Supervised Learning
• We have discussed how supervised learning is dealing with labeled data, but what are the types of supervised
learning and how it works?

• We will discuss a model called liner Regression to demonstrate how the models work.
Mathematics Revision
• Linear relation:
• Error value(e) =
• Residual Sum of Squares(RSS)=+ ………. the meaning of RSS carries a value that indicates the variances in
the error of a regression model.
• Optimization: optimization in a short is a method applied upon functions to find the minimum value of this
function. By taking the first derivative and find the values of variables that minimizes the first derivative (=0)
Loss Function
• Loss function: a loss function or cost function is a
function that maps an event or values of one or
more variables onto a real number intuitively
representing some "cost" associated with the event.
An optimization problem seeks to minimize a loss
function.
To further interpret: imagine if we have a certain
function that exists in a real time problem, we are
trying to get approximation of this function, we
calculate the error (loss) in at each point then we map
these values to find the equation parameters that
minimizes the loss function (find its minimum value)
Linear Regression
• Linear : a state that descries the relation of two variables, in this example x is linearly related to y.

X+Y=5
5
4
3
2
1
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
• Regression: a measure of the relationship between the mean value of one variable (output of your model) and
corresponding values of other variables (input features).
Cont.…
• Linear regression is a model that tries to find a linear equation that best represent the relation between two
variables.
• The general equation of linear Regression is:

RSS()=
The model will use the data samples to find the minimum values of thetas that best represent the relation
between the data points in linear relation
Cont.…
• What if the data contain nonlinear relation?
• We can extend the degree of the regression to fit more functions, but it adds in the model complexity and
calculations.
+
More ever if the relation involves more than one variable we can apply the same formula but in matrix form
nothing changes.
+
Where is a vector contain all the components
Cont.…
• Up till now its an optimization problem where is the learning happening ?
• Now we want to optimize the equation(RSS) to find the values of theta, but we don’t know theta
Let's put initially with any value,
We want the model to learn how to get this
We find the minimum by getting the first derivative of RSS with respect to
Then we decay the value of theta in each iteration by factor =
is called learning rate, (the step the model takes each iteration to find the minimum)
The model continues till it meets a certain evaluation criteria then it stops the learning process.
Cont.…
• Evaluation techniques and hazards of leaning will be covered in the next video inshallah.

• And mathematical derivation of the optimization of the parameters will be attached with the next video
inshallah.
Thanks

You might also like