Scribd
Upload
40 views5 pages

ML 07 Regularization

Regularization is the process of introducing additional information to solve problems more easily or reduce overfitting. It can enhance prediction accuracy and model interpretability. Regularization often encourages simpler models rather than more complex ones by adding a regularization term to the cost function that determines how much the regularization affects the solution.

Uploaded by

yiqing he
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
40 views5 pages

ML 07 Regularization

Regularization is the process of introducing additional information to solve problems more easily or reduce overfitting. It can enhance prediction accuracy and model interpretability. Regularization often encourages simpler models rather than more complex ones by adding a regularization term to the cost function that determines how much the regularization affects the solution.

Uploaded by

yiqing he
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Introduction to Regularization

The City College of New York


CSc 59929 -- Introduction to Machine Learning
Spring 2020 -- Erik K. Grimmelmann, Ph.D.
Regularization

Regularization is the process of introducing additional


information in order to solve problem more easily or to reduce
overfitting.

Regularization can enhance the prediction accuracy and/or


interpretability of the model it produces.

Regularization often encourages (or requires) models to be


simpler rather than more complex.

The City College of New York


CSc 59929 -- Introduction to Machine Learning 2
Spring 2020 -- Erik K. Grimmelmann, Ph.D.
Regularization
Regularization is accomplished by adding an additional
component, R ( w ) , the regularization term, to the basic cost
function, J 0 ( w ) . .

( w ) J0 ( w ) + λ R ( w )
J=

The scalar λ determines how much the regularization term


affects the solution that results when the cost function is
minimized.

The City College of New York


CSc 59929 -- Introduction to Machine Learning 3
Spring 2020 -- Erik K. Grimmelmann, Ph.D.
Regularization
The "length" of the w vector, as measured by a vector norm,
is often used as the basis for the regularization function, R ( w ) .

Different choices for the vector norm push the solution in


different directions.
n n
R ( w=
) L ( w ) − w=
2 2
0
2
i
2
0
=i 0=i 1
∑ w − w= ∑ i
w 2

favors solutions with smaller weights.


( w ) L0 ( w ) − w0
R=
favors solutions with fewer non-zero weights.
The City College of New York
CSc 59929 -- Introduction to Machine Learning 4
Spring 2020 -- Erik K. Grimmelmann, Ph.D.
Regularization

Regularization is the process of introducing additional


information in order to solve an ill-posed problem of to prevent
overfitting.

Regularization can enhance the prediction accuracy and/or


interpretability of the model it produces.

Regularization often encourages (or requires) models to be


simpler rather than more complex.

The City College of New York


CSc 59929 -- Introduction to Machine Learning 5
Spring 2020 -- Erik K. Grimmelmann, Ph.D.

You might also like