Scribd
Upload
25 views10 pages

Chapter5 Regularization Summary Final

The document discusses common machine learning model problems like underfitting and overfitting. Underfitting occurs when a model is too simple to capture the complexity in the training data, while overfitting occurs when a model learns the training data too well and fails to generalize to new data. The tradeoff between model complexity and accuracy can be addressed using techniques like regularization, which reduce complexity to prevent overfitting.

Uploaded by

Zubair Najim
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
25 views10 pages

Chapter5 Regularization Summary Final

The document discusses common machine learning model problems like underfitting and overfitting. Underfitting occurs when a model is too simple to capture the complexity in the training data, while overfitting occurs when a model learns the training data too well and fails to generalize to new data. The tradeoff between model complexity and accuracy can be addressed using techniques like regularization, which reduce complexity to prevent overfitting.

Uploaded by

Zubair Najim
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 10

The most common problems that can occur

while training a ML model

Underfitting Overfitting

When data lacks When data is too


complexity complex

1
When does underfitting happen?

A model is too simple or lacks complexity

A model is unable to find the patterns in the training data

A model generates a high error on the training set & unseen data

2
The inability of the model to understand
complexity of data: bias

Underfitting models can also be referred


as “highly biased”:

• A very simple straight line that does


not fit the data properly
• A large portion of the dataset is
ignored

The model performance is poor

3
When does overfitting happen?

The model is trained too much on a specific training dataset

The training data is very specific and has too many features

The model is unable to generalize testing data; showing low accuracy

4
The sensitivity of a model to a specific
dataset: variance

Overfitting models can also be referred as


“high variance models”:

• A very complex line is fitting each


datapoint but fails to recognize the
general pattern

The model is unable to make


accurate predictions on new data

5
The aims is to achieve a good balance
between the bias and the variance

• The performance of the model is affected by both


variance and bias which can lead to underfitting and
overfitting and eventually cause poor predictions.

• By adjusting variance and bias, we can generalize the


model so that it is neither too complex nor too simple.

6
The trade-off between bias and variance

• As variance increases
bias decreases

• As bias increases
variance decreases

7
How can we solve overfitting and
underfitting?

To solve underfitting To solve overfitting

making the data more making the data less


complex by increasing the complex by removing
number of observations in complexities
the training set & adding
new features

8
We can use regularization to reduce
complexity

How does regularization work?

Regularization shrinks coefficients towards zero,


so that the impact of less significant features is
reduced, and high variance is prevented.

9
Regularization uses loss functions:
L1 and L2

L1 L2

• Used in lasso regression • Used in ridge regression

• Less common • More common


• Not affected by outliers as it is • Not useful on dataset with
just considering the difference outliers as it is taking the
between actual and predicted squared difference which
values will increase the error

10

You might also like