.png)
Greedy Approach
Decision Trees build themselves using a Greedy Approach - at every node, they pick the split that looks best right now, without considering how it might affect splits further down the tree.
What "Greedy" Means Here
At each node:
1. Evaluate all possible splits
2. Pick the split with the highest Information Gain (or lowest Gini impurity)
3. Repeat this process independently for each child nodeWhy Trees Use a Greedy Strategy
Finding the globally optimal tree - the one with the absolute best combination of splits - is computationally infeasible for anything but tiny datasets, since the number of possible trees grows extremely fast.
The Trade-off
A greedy split that looks best immediately isn't always part of the best possible overall tree - a slightly worse split now might have enabled much better splits later, but the greedy algorithm can't see that far ahead.
Techniques like pruning and ensemble methods (covered next) exist partly to compensate for the short-sightedness of the greedy tree-building process.
Coming Up Next
Next, you'll be introduced to Ensemble Learning, a way of combining multiple models for stronger predictions.