Boosting

Boosting is a machine learning technique for perform supervised learning. Boosting occurs in stages, by incrementally adding to the current learned function. At every stage, a weak learner (i.e., one that can have an accuracy as bad as slightly greater than chance) is trained with the data. The output of the weak learner is then added to the learned function, with some strength (proportional to how accurate the weak learner is). Then, the data is reweighted: examples that the current learned function get wrong are "boosted" in importance, so that future weak learners will attempt to fix the errors.

There are several different boosting algorithms, depending on the exact mathematical form of the strength and weight. One of the most common boosting algorithms is AdaBoost[?]. Most boosting algorithms fit into the AnyBoost[?] framework, which shows that boosting performs gradient descent in function space[?].

Boosting is based on probably approximately correct learning[?], which is a branch of learning theory.

Algorithmically, boosting is related to

• logistic regression
• maximum entropy methods
• neural networks
• support vector machines.

External Links

• http://www.boosting.org/
• The boosting approach to machine learning: An overview (http://citeseer.nj.nec.com/489339)
• The AdaBoost algorithm (http://kiew.cs.uni-dortmund.de:8001/mlnet/instances/81d91e8d-dc15ed23e9)