Scribd
Upload
199 views1 page

The Problem of Overfitting - Coursera

The document discusses the problem of overfitting in machine learning models. It provides examples showing that while adding more features can improve a model's fit to training data, it can also result in overfitting where the model learns the random noise in the training data and fails to generalize to new examples. Underfitting occurs when the model is too simple to capture patterns in the data, while overfitting happens when the model is too complex and fits the random noise. The main ways to address overfitting are reducing the number of features or collecting more training data.

Uploaded by

Brian Ramiro Oporto Quispe
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
199 views1 page

The Problem of Overfitting - Coursera

The document discusses the problem of overfitting in machine learning models. It provides examples showing that while adding more features can improve a model's fit to training data, it can also result in overfitting where the model learns the random noise in the training data and fails to generalize to new examples. Underfitting occurs when the model is too simple to capture patterns in the data, while overfitting happens when the model is too complex and fits the random noise. The main ways to address overfitting are reducing the number of features or collecting more training data.

Uploaded by

Brian Ramiro Oporto Quispe
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

The Problem of Overfitting | Coursera https://www.coursera.org/learn/machine-learning/supplement/VTe37/th...

Volver a la semana 3 Lecciones This Course: Aprendizaje Automático Anterior Siguiente

The Problem of Overfitting


Consider the problem of predicting y from x ∈ R. The leftmost figure below shows
the result of fitting a y = θ0 + θ1 x to a dataset. We see that the data doesn’t really
lie on straight line, and so the fit is not very good.

Instead, if we had added an extra feature x2 , and fit y = θ0 + θ1 x + θ2 x2 , then


we obtain a slightly better fit to the data (See middle figure). Naively, it might seem
that the more features we add, the better. However, there is also a danger in
adding too many features: The rightmost figure is the result of fitting a 5th order
5
polynomial y = ∑j=0 θj xj . We see that even though the fitted curve passes
through the data perfectly, we would not expect this to be a very good predictor
of, say, housing prices (y) for different living areas (x). Without formally defining
what these terms mean, we’ll say the figure on the left shows an instance of
underfitting—in which the data clearly shows structure not captured by the
model—and the figure on the right is an example of overfitting.

Underfitting, or high bias, is when the form of our hypothesis function h maps
poorly to the trend of the data. It is usually caused by a function that is too simple
or uses too few features. At the other extreme, overfitting, or high variance, is
caused by a hypothesis function that fits the available data but does not generalize
well to predict new data. It is usually caused by a complicated function that creates
a lot of unnecessary curves and angles unrelated to the data.

This terminology is applied to both linear and logistic regression. There are two
main options to address the issue of overfitting:

1) Reduce the number of features:

Manually select which features to keep.

Use a model selection algorithm (studied later in the course).

1 de 1 31/05/2018 2:51

You might also like