.png)
Boosting
Unlike Bagging, which trains models independently, Boosting builds models sequentially - each new model focuses on correcting the mistakes made by the ones before it.
How Boosting Works
1. Train an initial model on the data
2. Identify the samples it got wrong
3. Train the next model with more emphasis on those hard samples
4. Repeat, combining all models into a weighted final predictionImplementing in Python
from sklearn.ensemble import GradientBoostingClassifier
model = GradientBoostingClassifier(
n_estimators=100,
learning_rate=0.1,
random_state=42
)
model.fit(X_train, y_train)
print(model.predict(X_test))Popular Boosting Algorithms
AdaBoost was one of the earliest boosting methods, while Gradient Boosting and its faster variants like XGBoost and LightGBM are widely used today for their strong predictive performance.
Because boosting keeps correcting errors aggressively, it can overfit if run for too many iterations - the learning_rate and n_estimators parameters help control this.
Coming Up Next
Next, you'll learn about Random Forest, one of the most popular bagging-based ensembles.