Logistic Regression: Classification
Logistic Regression: Classification
Regression
Classification
Machine Learning
Slides from CS-229 by Andrew Ng
Classification
Andrew Ng
(Yes) 1
Malignant ?
(No) 0
Tumor Size Tumor Size
Logistic Regression:
Andrew Ng
Logistic
Regression
Hypothesis
Representation
Machine Learning
Logistic Regression Model
Want
0.5
Sigmoid function 0
Logistic function
Andrew Ng
Interpretation of Hypothesis Output
= estimated probability that y = 1 on input x
Example: If
Andrew Ng
Logistic
Regression
Decision boundary
Machine Learning
Logistic regression 1
0.5
0
z
Suppose predict “ “ if
Alternatively, if 𝑧 ≥ 0 ; 𝜃𝑇 𝑥 ≥ 0
predict “ “ if
Alternatively, if 𝑧 <0 ; 𝜃𝑇 𝑥< 0
Andrew Ng
Decision Boundary
x2 Linear decision boundary
3
2
1
Predict “ “ if 𝜃𝑇 𝑥 ≥ 0
1 2 3 x1
Suppose
Predict “ “ if
𝑥1 + 𝑥 2 ≥ 3
Andrew Ng
Logistic
Regression
Cost function
Machine Learning
Training set:
m examples
“non-convex” “convex”
Andrew Ng
Logistic regression cost function
If y = 1
0 1 Andrew Ng
Logistic regression cost function
If y = 0
0 1 Andrew Ng
Logistic
Regression
Simplified cost function
Machine Learning
Logistic regression cost function
𝐶𝑜𝑠𝑡
( h𝜃 (𝑥 ) , 𝑦 ) =− 𝑦𝑙𝑜𝑔 ( h𝜃 ( 𝑥 ) ) − (1 − 𝑦 ) log (1 −h 𝜃 ( 𝑥 ))
Andrew Ng
Logistic regression cost function
To fit parameters :
Andrew Ng
Gradient Descent
Want :
Repeat
Andrew Ng
Gradient Descent
Want :
Repeat
Andrew Ng
Gradient Descent
Want :
Repeat
Want :
Repeat
Machine Learning
Multiclass classification
Email tagging: Work, Friends, Family, Hobby, etc.
Andrew Ng
Binary classification: Multi-class classification:
x2 x2
x1 x1
Andrew Ng
x2
One-vs-all (one-vs-rest):
x1
x2 x2
x1 x1
x2
Class 1:
Class 2:
Class 3:
x1
Andrew Ng
One-vs-all
Andrew Ng