Scribd
Upload
45 views

The Boosting Algorithm Adaboost

Boosting is a machine learning meta-algorithm that can take weak learners and combine them into a strong learner. Specifically, AdaBoost is an algorithm that can be used with other learning algorithms to improve their performance by combining their outputs into a weighted sum. AdaBoost tweaks subsequent weak learners to focus on instances that previous learners misclassified. As long as each weak learner performs slightly better than random guessing, the final model converges to a strong learner.

Uploaded by

Arnav Mendi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
45 views

The Boosting Algorithm Adaboost

Boosting is a machine learning meta-algorithm that can take weak learners and combine them into a strong learner. Specifically, AdaBoost is an algorithm that can be used with other learning algorithms to improve their performance by combining their outputs into a weighted sum. AdaBoost tweaks subsequent weak learners to focus on instances that previous learners misclassified. As long as each weak learner performs slightly better than random guessing, the final model converges to a strong learner.

Uploaded by

Arnav Mendi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 2

Boosting

Boosting is a machine learning ensemble meta-algorithm for


primarily reducing bias, and also variance[1] in supervised learning,
and a family of machine learning algorithms which convert weak
learners to strong ones.[2] Boosting is based on the question posed by
Kearns and Valiant (1988, 1989):[3][4] Can a set of weak learners
create a single strong learner? A weak learner is defined to be a
classifier which is only slightly correlated with the true classification (it
can label examples better than random guessing). In contrast, a strong
learner is a classifier that is arbitrarily well-correlated with the true
classification.
AdaBoost
AdaBoost can be used in conjunction with many other types of
learning algorithms to improve their performance. The output of the
other learning algorithms ('weak learners') is combined into a weighted
sum that represents the final output of the boosted classifier. AdaBoost
is adaptive in the sense that subsequent weak learners are tweaked in
favor of those instances misclassified by previous classifiers. AdaBoost
is sensitive to noisy data and outliers. In some problems, however, it
can be less susceptible to the overfitting problem than other learning
algorithms. The individual learners can be weak, but as long as the
performance of each one is slightly better than random guessing (e.g.,
their error rate is smaller than 0.5 for binary classification), the final
model can be proven to converge to a strong learner

The boosting algorithm AdaBoost

[1] Explaining AdaBoost, Robert E. Schapire

[2] http://en.wikipedia.org/wiki/AdaBoost
[3] https://en.wikipedia.org/wiki/Boosting_(machine_learning)

You might also like