Voting and Averaging

Voting and Averaging are the simplest ways to combine predictions from multiple models in an ensemble, without needing a meta-model like in stacking.

Voting (Classification)

Hard Voting:  Each model casts a vote for a class; majority wins
Soft Voting:  Each model outputs class probabilities; the ensemble
              averages them and picks the class with the highest average

Averaging (Regression)

FinalPrediction = (Model1.predict(x) + Model2.predict(x) + Model3.predict(x)) / 3

# Can also be a weighted average, giving stronger models more influence

Implementing in Python

from sklearn.ensemble import VotingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import SVC

voting_model = VotingClassifier(estimators=[
    ("lr", LogisticRegression()),
    ("dt", DecisionTreeClassifier()),
    ("svc", SVC(probability=True))
], voting="soft")

voting_model.fit(X_train, y_train)
print(voting_model.predict(X_test))

When to Use Which

Soft voting generally performs better than hard voting when the models produce reliable probability estimates, since it accounts for how confident each model is, not just its final label.

Coming Up Next

Next, you'll shift gears to Time Series Analysis, a different area of data science focused on data ordered over time.