Decision Tree Classifier

A Decision Tree Classifier predicts outcomes by learning a series of if-else questions about the features, forming a flowchart-like structure that leads to a final class label.

How It Works

Is Income > 50000?
+-- Yes: Is Age > 30?
|         +-- Yes: Class = "Buys"
|         +-- No:  Class = "Doesn't Buy"
+-- No:  Class = "Doesn't Buy"

Implementing in Python

import pandas as pd
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split

df = pd.DataFrame({
    "Income": [20000, 25000, 40000, 45000, 60000, 65000, 80000, 90000],
    "Age": [22, 45, 25, 50, 28, 55, 30, 35],
    "Buys": [0, 0, 0, 1, 1, 1, 1, 1]
})

X = df[["Income", "Age"]]
y = df["Buys"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = DecisionTreeClassifier(max_depth=3, random_state=42)
model.fit(X_train, y_train)

print(model.predict(X_test))

Splitting Criteria

At each step, the tree chooses the feature and threshold that best separates the classes, commonly measured using Gini impurity or entropy (information gain).

Coming Up Next

Next, you'll take a closer look at the individual nodes that make up a Decision Tree.