Scribd
Upload
17 views

Lecture 08 ML

The document discusses different machine learning concepts and algorithms for training models. It covers topics like training models, computational complexity, gradient descent including batch, stochastic, and mini-batch variants, and polynomial regression. Gradient descent is described as an iterative algorithm for minimizing a cost function by adjusting parameters in the direction of the negative gradient.

Uploaded by

saharabdouma
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
17 views

Lecture 08 ML

The document discusses different machine learning concepts and algorithms for training models. It covers topics like training models, computational complexity, gradient descent including batch, stochastic, and mini-batch variants, and polynomial regression. Gradient descent is described as an iterative algorithm for minimizing a cost function by adjusting parameters in the direction of the negative gradient.

Uploaded by

saharabdouma
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Training Models

Mohamed Yasser
1) Training model
2) Computational complexity
3) Gradient descent
4) Batch gradient descent
Topics 5) Stochastic gradient
descent
6) Mini batch gradient
descent
7) Polynomial Regression

9/3/20XX Presentation Title 2


Training model
training a model refers to the process of
teaching a machine learning algorithm
to make predictions or decisions based
on input data. The goal is to enable the
model to generalize well to new, unseen
data.

9/3/20XX Presentation Title 3


Computational complexity
In machine learning, computational
complexity refers to the amount of
computational resources, such as time
and memory, required by an algorithm
to perform a certain task as a function of
the input size. It is concerned with
analyzing the efficiency and scalability
of machine learning algorithms.
Example :
- KNN algorithm it take O(n log n) .
9/3/20XX Presentation Title 4
Linear model
Linear regression is a statistical method and a
fundamental algorithm in machine learning used for
modeling the relationship between a dependent variable
(target) and one or more independent variables
(features). The basic idea is to find the best-fitting linear
relationship (a straight line) that represents the data. This
relationship is expressed in the form of a linear equation.
The equation for simple linear regression with one
independent variable is:
Y = mx + b
Y : dependent variable
X : independent variable
M : slope
B : bias 5
Gradient descent
Gradient Descent is an iterative optimization
algorithm commonly used in machine learning to
minimize a cost function. Its primary purpose is to find
the optimal parameters for a model by adjusting
them in the direction that leads to a reduction in the
cost.
The general idea behind Gradient Descent is to
iteratively move towards the minimum of a function
by taking steps proportional to the negative of the
gradient (derivative) of the function at the current
point.

7
Formula

θ: The parameter or weight being


optimized.

α: The learning rate, which is a


positive constant determining the
step size of the update.

The partial derivative of the


cost function J(θ)
8
How it work ..!
1. Initialize weights randomly.
2. Calculate the predicted output.
3. Compute the loss between predicted and actual
output.
4. Calculate the gradient of the loss with respect to
the weights.
5. Update weights using the gradient and a learning
rate.
6. Repeat steps 2-5 until convergence or a specified
number of iterations.

9
Types of gradient descent

✓ Batch Gradient Descent (BGD)

✓ Stochastic Gradient Descent (SGD)

✓ Mini-Batch Gradient Descent:

10
Batch gradient descent
In Batch Gradient Descent, the entire
training dataset is used to compute the
gradient of the cost function with respect
to the model parameters in each iteration.
The algorithm calculates the average
gradient for the entire dataset and then
updates the model parameters.

11
Stochastic gradient descent
In Stochastic Gradient Descent, the model
parameters are updated after computing
the gradient of the cost function with
respect to the parameters for each
training example individually. The key
difference from Batch Gradient Descent is
that it uses only one training example at a
time for the update.

13
Mini batch gradient descent
Mini-Batch Gradient Descent is a
compromise between Batch Gradient
Descent and Stochastic Gradient Descent.
Instead of using the entire training
dataset (Batch GD) or just one example
(SGD), Mini-Batch GD processes a small
random subset of the training data (a
mini-batch) in each iteration.

15
Polynomial regression

Polynomial regression is a type of


regression analysis in which the
relationship between the independent
variable x and the dependent variable y is
modeled as an n-th degree polynomial.
Instead of fitting a straight line (as in linear
regression), polynomial regression uses a
polynomial equation to capture the
nonlinear relationships between variables.

17
Thank you

20

You might also like