Random Forest

A Random Forest is an ensemble of many Decision Trees, built using bagging along with an extra layer of randomness - each tree only considers a random subset of features at every split.

Why Add Feature Randomness

# Regular bagging alone can still produce correlated trees
# if one feature is very strong.
#
# Random Forest fixes this by making each split choose from
# only a random subset of features, forcing trees to diversify.

Implementing in Python

import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

df = pd.DataFrame({
    "Income": [20000, 25000, 40000, 45000, 60000, 65000, 80000, 90000],
    "Age": [22, 45, 25, 50, 28, 55, 30, 35],
    "Buys": [0, 0, 0, 1, 1, 1, 1, 1]
})

X = df[["Income", "Age"]]
y = df["Buys"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

print(model.predict(X_test))
print("Feature importances:", model.feature_importances_)

Why Random Forest Works Well

By combining many diverse, decorrelated trees, Random Forest reduces the overfitting tendency of a single Decision Tree while keeping much of its interpretability through feature importance scores.

Coming Up Next

Next, you'll learn about Stacking, an ensemble technique that combines different types of models.