Scribd
Upload
168 views

AdaBoostExample PDF

The document summarizes an example of AdaBoosting applied to a dataset with 10 data points to classify as either 1 or -1. AdaBoost iteratively selects weak learners (classifiers) that best classify the reweighted data at each stage. Over 3 iterations, it arrives at a composite classifier with 0 errors on the training data using three weak learners based on thresholds of x < 2.5, x < 8.5, and x > 5.5. The probabilities of each data point are updated after each stage to increase the weight of misclassified points for the next iteration.

Uploaded by

chongj2
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
168 views

AdaBoostExample PDF

The document summarizes an example of AdaBoosting applied to a dataset with 10 data points to classify as either 1 or -1. AdaBoost iteratively selects weak learners (classifiers) that best classify the reweighted data at each stage. Over 3 iterations, it arrives at a composite classifier with 0 errors on the training data using three weak learners based on thresholds of x < 2.5, x < 8.5, and x > 5.5. The probabilities of each data point are updated after each stage to increase the weight of misclassified points for the next iteration.

Uploaded by

chongj2
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Example of AdaBoosting

The training data:


index: 0 1 2 3 4 5 6 7 8 9
x value: 0 1 2 3 4 5 6 7 8 9
y value: 1 1 1 -1 -1 -1 1 1 1 -1
The weak learner produces hypotheses of the form: x < v, or x > v. The threshold v is
determined to minimize the probability of error over the entire data. (No sampling.)

Running the algorithm.


We start with the following probabilities:
p0 p1 p2 p3 p4 p5 p6 p7 p8 p9
0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
t = 1. The best threshold is between 2 and 3.
h1 (x) = I(x < 2.5)
1 = 0.3
1 = 0.423649
qi = 1.52753 for wrong, 0.654654 for right
Updating the probabilities:

index: 0 1 2 3 4 5 6 7 8 9
correct: y y y y y y n n n y
old pi 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
pre-normalized pi .06547 .06547 .06547 .06547 .06547 .06547 .15 .15 .15 .06547
Z1 = 0.916515
new pi .07143 .07143 .07143 .07143 .07143 .07143 .16667 .16667 .16667 .07143

f1 (x) = 0.423649 I(x < 2.5), 3 mistakes

t = 2. Now a threshold between 2 and 3 gives error of 0.5, the threshold between 5 and 6 gives
0.28, the threshold between 8 and 9 gives 0.214, best.
h2 (x) = I(x < 8.5)
2 = 0.214
2 = 0.6496
Updating the probabilities:

index: 0 1 2 3 4 5 6 7 8 9
correct: y y y n n n y y y y
pre-normalized pi .037 .037 .037 .137 .137 .137 .087 .087 .087 .037
Z2 = 0.82
new pi .045 .045 .045 .167 .167 .167 .106 .106 .106 .045
f2 (x) = 0.423649 I(x < 2.5) + 0.6496 I(x < 8.5), 3 mistakes

t = 3. The best threshold is between 5 and 6.

h3 (x) = I(x > 5.5)


3 = 0.1818
3 = 0.7520

Updating the probabilities:

index: 0 1 2 3 4 5 6 7 8 9
correct: n n n y y y y y y n
pre-normalized pi .0964 .0964 .0964 .078 .078 .078 .05 .05 .05 .0964
Z3 = 0.77139
new pi .125 .125 .125 .102 .102 .102 .064 .064 .064 .125

f3 (x) = 0.423649 I(x < 2.5) + 0.6496 I(x < 8.5) + 0.752 I(x > 5.5), 0 mistakes

You might also like