Bagging

Bagging, short for Bootstrap Aggregating, is an ensemble technique that trains multiple copies of the same model on different random samples of the training data, then combines their predictions.

How Bagging Works

1. Create several random samples of the training data (with replacement)
2. Train one model on each sample independently
3. Combine predictions - majority vote for classification,
   average for regression

Implementing in Python

from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier

model = BaggingClassifier(
    estimator=DecisionTreeClassifier(),
    n_estimators=50,
    random_state=42
)

model.fit(X_train, y_train)
print(model.predict(X_test))

Why Bagging Reduces Variance

Since each model sees a slightly different sample of the data, their individual errors tend to differ - averaging or voting across them smooths out those errors and produces a more stable overall prediction.

Coming Up Next

Next, you'll learn about Boosting, a different ensemble strategy that builds models sequentially.