Scribd
Upload
9 views

Lecture 21 - Logistic Regression

Uploaded by

ilham.hasib
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
9 views

Lecture 21 - Logistic Regression

Uploaded by

ilham.hasib
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 34

21 Logistic Regression

(Slides Adapted from


Andrew Ng’s slides)
- Dr. Sifat Momen (SfM1)
Learning goals
• After this presentation, you should be able to
• Understand how linear regression is used to solve classification problem
• Understand the underlying concept of logistic regression in the realms of
binary classification problem
• The limitation of linear regression to solving classification problems and how
are these problems resolved in logistic regression
• The OVR technique to extend the solution to multi-class classification
problem

08/06/2024 Slides by Dr. Sifat Momen 2


Classification

Email: Spam / Not Spam?


Online Transactions: Fraudulent (Yes / No)?
Tumor: Malignant / Benign ?

0: “Negative Class” (e.g., benign tumor)


1: “Positive Class” (e.g., malignant tumor)
(Yes) 1

Malignant ?

(No) 0
Tumor Size Tumor Size
Threshold classifier output at 0.5:
If , predict “y = 1”
If , predict “y = 0”
(Yes) 1

Malignant ?

(No) 0
Tumor Size Tumor Size
Classification: y = 0 or 1

can be > 1 or < 0

Logistic Regression:
Logistic
Regression
Hypothesis
Representation
Machine Learning
Logistic Regression Model
Want

0.5

Sigmoid function 0

Logistic function
Interpretation of Hypothesis Output
= estimated probability that y = 1 on input x

Example: If

Tell patient that 70% chance of tumor being malignant

“probability that y = 1, given x,


parameterized by ”
Logistic
Regression
Decision boundary

Machine Learning
Logistic regression 1

z
Suppose predict “ “ if

predict “ “ if
Decision Boundary
x2
3
2

1 2 3 x1

Predict “ “ if
Decision Boundary
x2
3
2

1 2 3 x1

Predict “ “ if
Non-linear decision boundaries
x2

-1 1 x1
-1

x2
Non-linear decision boundaries
x2

-1 1 x1
-1
Predict “ “ if
x2
Non-linear decision boundaries
Logistic
Regression
Cost function

Machine Learning
Training set:

m examples

How to choose parameters ?


Cost function
Linear regression:

“non-convex” “convex”
Logistic regression cost function

If y = 1

0 1
Logistic regression cost function

If y = 0

0 1
Logistic
Regression
Simplified cost function
and gradient descent

Machine Learning
Logistic regression cost function
Logistic regression cost function

To fit parameters :

To make a prediction given new :


Output
Gradient Descent

Want :
Repeat

(simultaneously update all )


Gradient Descent

Want :
Repeat

(simultaneously update all )

Algorithm looks identical to linear regression!


Logistic
Regression
Multi-class classification:
One-vs-all

Machine Learning
Multiclass classification
Email foldering/tagging: Work, Friends, Family, Hobby

Medical diagrams: Not ill, Cold, Flu

Weather: Sunny, Cloudy, Rain, Snow


Binary classification: Multi-class classification:

x2 x2

x1 x1
x2
One-vs-all (one-vs-rest):

x1
x2 x2

x1 x1
x2
Class 1:
Class 2:
Class 3:
x1
One-vs-all

Train a logistic regression classifier for each class


to predict the probability that .

On a new input , to make a prediction, pick the


class that maximizes
Sklearn Implementation

https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html

You might also like