Scribd
Upload
38 views

ML Logistic Regression

This document summarizes lectures on logistic regression machine learning models. It introduces logistic regression, explaining that it is a supervised classification model best for binary problems that uses the sigmoid function. It describes how logistic regression calculates the probability of an output being 1 using weights, features, and bias in the sigmoid function. The document also discusses gradient descent for optimizing logistic regression, the logistic regression loss function, and provides an example calculation.

Uploaded by

Zarfa Masood
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
38 views

ML Logistic Regression

This document summarizes lectures on logistic regression machine learning models. It introduces logistic regression, explaining that it is a supervised classification model best for binary problems that uses the sigmoid function. It describes how logistic regression calculates the probability of an output being 1 using weights, features, and bias in the sigmoid function. The document also discusses gradient descent for optimizing logistic regression, the logistic regression loss function, and provides an example calculation.

Uploaded by

Zarfa Masood
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 19

CS446: Machine Learning

Lecture 16-17 (ML Models – Logistic Regression)


Instructor:
Dr. Muhammad Kabir
Assistant Professor
[email protected]

School of Systems and Technology


Department of Computer Science
University of Management and Technology, Lahore
Previous Lectures…
 Linear Regression – intuition

 Linear Regression – Mathematical understanding

 Gradient Decent for Linear Regression

 Implementation of Linear Regression


Today’s Lectures…
 Logistic Regression – intuition

 Logistic Regression – Mathematical understanding

 Gradient Decent for Logistic Regression

 Implementation of Logistic Regression


How Machine Learning Works….
Logistic Regression - Intuition….
 Supervised Learning
model
 Classification model
 Best for binary
classification problem
 It uses sigmoid function.
 Binary cross entropy loss
function (or) Log loss.
Logistic Regression - Intuition….
𝑌෠ − 𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑡ℎ𝑎𝑡 (𝑦 = 1)
𝑌෠ − 𝑃 𝑌 = 1 𝑋)
X – input features
w – weights (number of
weights is equal to the number
of input features in a dataset)
b – bias
𝑌෠ = 𝜎(𝑍)

1 Where
෠
𝑌= Large –ve values will be closer to
1 + 𝑒 −𝑍 Z = w.X + b
zero and large +ve values will be
Sigmoid function Suppose Z=5X+10 closer to 1
Logistic Regression
Advantages
 Easy to implement.
 Perform well on data with linear relationship.
 Less prone to overfitting for low dimensional dataset.

Disadvantages
 High dimensional dataset causes overfitting.
 Difficult to capture complex relationship in a dataset.
 Sensitive to outliers.
 Needs a larger dataset.
Logistic Regression – Example Calculation….

1
Suppose Z=5X+10 𝑌෠ = Sigmoid function
1+𝑒 −𝑍
Logistic Regression - Inference….
If Z value is large positive
number, then
1
𝑌෠ =
1+0
𝑌෠ = 1

If Z value is large negative


number, then
1
𝑌෠ =
1+𝑙𝑎𝑟𝑔𝑒 𝑝𝑜𝑠𝑖𝑡𝑣𝑒 𝑛𝑢𝑚𝑏𝑒𝑟
1 𝑌෠ = 0
𝑌෠ =
1 + 𝑒 −𝑍
Sigmoid function
Loss Function - Logistic Regression….
Loss Function: Loss function measures how far the estimated(predicted)
value is from the its true value. 𝑛
1 2
𝐿𝑜𝑠𝑠 = ෍ 𝑌𝑖 − 𝑌෡𝑖
𝑛
𝑖=1
Loss Function - Logistic Regression….
Binary Cross Entropy Loss function (or) Log Loss:

Two possible values (0 or 1)


Loss Function - Logistic Regression….
Binary Cross Entropy Loss function (or) Log Loss:

Two possible values (0 or 1)


Loss Function - Logistic Regression….
Binary Cross Entropy Loss function (or) Log Loss:

Two possible values (0 or 1)


Cost Function - Logistic Regression….
Loss function (L) mainly applies for a single training set as compared to the
cost function (J) which deals with a penalty for a number of training sets or
the complete batch.

Loss function (L)

‘m’ represents the number of data points in the training set Cost function (J)
Gradient Decent for Logistic Regression...
Working of Gradient Decent .
Gradient Descent - Logistic Regression
- Gradient decent is an optimization algorithm for minimizing
the cost function in various ML algorithms.
- It is used for updating the parameters of the learning
model.
- Formula for updating w and b is:
- w --> weight
- b --> bias
- L --> Learning rate
- dw -->partial derivative of loss unction with respect to w.
- db --> partial derivate of loss function with respect to b.
Logistic Regression

- Things required for Logistic


Regression

- Sigmoid function
- Updating weights through Gradient
decent
- Derivate to get dw and db.
Chapter Reading

Chapter Chapter 01
Pattern Recognition and
Machine Learning
by
Machine Learning
by
Tom Mitchell
Christopher M. Bishop

You might also like