Scribd
Upload
88 views27 pages

1-Logistic Regression

Logistic regression is a classification algorithm used to predict a binary outcome like spam/not spam or malignant/benign. It models the probability of the outcome being 1 using the logistic function. The parameters are estimated to minimize the cost function, which is optimized using gradient descent. For multi-class problems, logistic regression can be extended to the "one-vs-all" approach, training a separate model for each class.

Uploaded by

abdala sabry
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
88 views27 pages

1-Logistic Regression

Logistic regression is a classification algorithm used to predict a binary outcome like spam/not spam or malignant/benign. It models the probability of the outcome being 1 using the logistic function. The parameters are estimated to minimize the cost function, which is optimized using gradient descent. For multi-class problems, logistic regression can be extended to the "one-vs-all" approach, training a separate model for each class.

Uploaded by

abdala sabry
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 27

Logistic

Regression
Classification
Machine Learning
Classification

Email: Spam / Not Spam?


Online Transactions: Fraudulent (Yes / No)?
Tumor: Malignant / Benign ?

0: “Negative Class” (e.g., benign tumor)


1: “Positive Class” (e.g., malignant tumor)

Andrew Ng
(Yes) 1

Malignant ?

(No) 0
Tumor Size Tumor Size

Threshold classifier output at 0.5:


If , predict “y = 1”
If , predict “y = 0”
Andrew Ng
Classification: y = 0 or 1

can be > 1 or < 0

Logistic Regression:

Andrew Ng
Logistic
Regression
Hypothesis
Representation
Machine Learning
For convenience of notation, define .

Multivariate linear regression.


Andrew Ng
Logistic Regression Model
Want

0.5

Sigmoid function 0

Logistic function
Andrew Ng
Interpretation of Hypothesis Output
= estimated probability that y = 1 on input x

Example: If

Tell patient that 70% chance of tumor being malignant

“probability that y = 1, given x,


parameterized by ”

Andrew Ng
Logistic
Regression
Decision boundary

Machine Learning
Logistic regression 1

z
Suppose predict “ “ if

predict “ “ if

Andrew Ng
Decision Boundary
x2
3
2

1 2 3 x1

Predict “ “ if

Andrew Ng
Non-linear decision boundaries
x2

-1 1 x1
-1
Predict “ “ if
x2

x1

Andrew Ng
Logistic
Regression
Cost function

Machine Learning
Training set:

m examples

How to choose parameters ?


Andrew Ng
Cost function
Linear regression:

“non-convex” “convex”

Andrew Ng
Logistic regression cost function

If y = 1

0 1 Andrew Ng
Logistic regression cost function

If y = 0

0 1 Andrew Ng
Logistic
Regression
Simplified cost function
and gradient descent

Machine Learning
Logistic regression cost function

Andrew Ng
Logistic regression cost function

To fit parameters :

To make a prediction given new :


Output

Andrew Ng
Gradient Descent

Want :
Repeat

(simultaneously update all )

Andrew Ng
Gradient Descent

Want :
Repeat

(simultaneously update all )

Algorithm looks identical to linear regression!


Andrew Ng
Logistic
Regression
Multi-class classification:
One-vs-all

Machine Learning
Multiclass classification
Email foldering/tagging: Work, Friends, Family, Hobby

Medical diagrams: Not ill, Cold, Flu

Weather: Sunny, Cloudy, Rain, Snow

Andrew Ng
Binary classification: Multi-class classification:

x2 x2

x1 x1
Andrew Ng
x2
One-vs-all (one-vs-rest):

x1
x2 x2

x1 x1
x2
Class 1:
Class 2:
Class 3:
x1
Andrew Ng
One-vs-all

Train a logistic regression classifier for each


class to predict the probability that .

On a new input , to make a prediction, pick the


class that maximizes

Andrew Ng

You might also like