Scribd
Upload
7 views4 pages

Bias and Variance

The document explains the concepts of bias and variance in machine learning, highlighting that bias refers to errors in training data while variance relates to errors in test data. It discusses underfitting, where a model fails to learn patterns adequately, and overfitting, where a model performs well on training data but poorly on test data. The document also introduces the bias-variance trade-off and suggests regularization as a method to manage both underfitting and overfitting.

Uploaded by

Shreya Parekh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7 views4 pages

Bias and Variance

The document explains the concepts of bias and variance in machine learning, highlighting that bias refers to errors in training data while variance relates to errors in test data. It discusses underfitting, where a model fails to learn patterns adequately, and overfitting, where a model performs well on training data but poorly on test data. The document also introduces the bias-variance trade-off and suggests regularization as a method to manage both underfitting and overfitting.

Uploaded by

Shreya Parekh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

11-08-2024

Machine Learning
Bias and Variance

Bias and Variance


• In a nutshell, Bias is the Error of the training data, while Variance is the
Error of the test data
• Bias:
• If a model is not learnt enough the training data, it is known as underfitting, and
the model is said to have high bias.
• The model could be too simple for given features or finding patterns in given
features could be difficult.
• Variance:
• Model is said to have high variance when model overfits training data
• Model learnt too precisely. High error in testing data.
• Model with high variance is likely to have learned noise in the training data rather
than focussing more on pattern.

1
11-08-2024

Bias and Variance

Bias and Variance

2
11-08-2024

Bias and Variance


• While reducing bias, the model
fits exactly well on a particular
sample of training data and is
unable to find the basic patterns
in the dataset that it has never
trained. So, the model can have
deviated outcomes while trained
on another sample, and hence
produce high variance.
• Similarly, if willing to keep minor
deviation or low variance when
distinct samples datasets are
used, then the model would not
fit exactly on data points that
lead to high bias.

Underfitting
• Model has not adequately learnt the patterns in the training dataset
• Therefore, Underfitted model
• is unable to correctly perform on new data
• performs poorly even on training data
• Generally, it occurs when both bias and variance are low

3
11-08-2024

Overfitting
• Model perform exceptionally well on training data but poorly on the test
data
• It might occur when it possessed low bias and high variance.
• Reasons:
• Possibility of a large number of features compared to the sample size; with the
many features, there are chances of developing spurious relationship between
features and target.
• It can be managed by regularization which modifies loss function.
• In fact, both under- and over-fitting can be handled using regularization
• End goal is to make less mistake in test data

Bias-Variance Trade off


• Bias-variance relationship

𝑀𝑆𝐸 = 𝐵𝑖𝑎𝑠 2 + 𝑉𝑎𝑟 + 𝜎𝜀2

• Derivation: https://towardsdatascience.com/the-bias-variance-tradeoff-8818f41e39e9

• Watch: https://www.youtube.com/watch?v=EuBBz3bI-aA

You might also like