Scribd
Upload
10 views12 pages

Week 06

The document discusses multiclass classification and regularization for linear and logistic regression models. It provides examples of multiclass classification problems including fruit classification, player categories, and weather prediction. It then describes using a one-vs-all approach for multiclass classification with logistic regression, where a separate logistic regression model is trained for each class to predict the probability that an observation belongs to that class. The document also discusses how regularization can help reduce overfitting for linear and logistic regression models by penalizing model complexity. Gradient descent equations are provided for regularization with both linear and logistic regression.

Uploaded by

Osii C
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
10 views12 pages

Week 06

The document discusses multiclass classification and regularization for linear and logistic regression models. It provides examples of multiclass classification problems including fruit classification, player categories, and weather prediction. It then describes using a one-vs-all approach for multiclass classification with logistic regression, where a separate logistic regression model is trained for each class to predict the probability that an observation belongs to that class. The document also discusses how regularization can help reduce overfitting for linear and logistic regression models by penalizing model complexity. Gradient descent equations are provided for regularization with both linear and logistic regression.

Uploaded by

Osii C
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 12

Multiclass

Classification
Multiclass Classification using LR
• Examples of multiclass classification:
• Classification of different fruits: apples, oranges, peaches, bananas
y= 1 2 3 4
• Category of players: gold, silver, platinum
y=1 2 3
• Weather: dry, rain, snow
y= 0 1 2
Multiclass Classification using LR
One vs All

(x)

(x)

(x)
One vs All

(x)

h ( 𝑥 ) =𝑃 ( 𝑦=𝑖| 𝑥 ; 𝜃 ) (𝑖=1 ,2 , 3)
𝑖
𝜃

(x)
• Train an LR classifier for each class i to predict the
probability that y = I
• On a new data point x, to make a prediction, choose
the class i that maximizes the hypothesis h

h1=0.8 (x)
H2 = 0.3
H3=0.55
Regularization
for Lin Reg
and Log Reg
Regularization for Linear
Regression
• Simple models are prone to underfitting (High bias)
• Not even good for training examples
• Complex models are prone to overfitting (High
variance)
• Too many features
• Works well with training examples
• Not good for test example (not generalizable)
• Example of housing price predictor
Regularization for Linear Regression
• Consider we have only one feature, that is the size of house to predict the price.
• Suppose we penalize and make very small.
• +
• Example:
• Features: x1, x2, …, x100
• Parameters: 0, 1, 2, …, 100

• In regularization, we select small values of , , … to avoid overfitting.


Regularization for Linear Regression
• Gradient Descent
𝑖
𝑥 𝑗

𝑋 =¿
𝑚
1 𝜆 2
𝜃2=𝜃 2 −𝛼 [ ∑ ( h 𝜃 ( 𝑥 ) − 𝑦 ) 𝑥2 + 𝜃 𝑗 ]
1 1 1
2 𝑖=1 𝑚
Regularization for Logistic Regression
• Gradient descent for Log Reg w/o regularization:

• Gradient descent for Log Reg with regularization:


𝑚
𝜃0 =𝜃 0 − 𝛼 ∑ ( h 𝜃 𝑥 − 𝑦 ) 𝑥0 ( 𝑗=0)
( 𝑖
) 𝑖 𝑖

𝑖=1

You might also like