Model Evaluation in Machine Learning
Model evaluation measures how well a trained model performs, both on the data it was trained on and — more importantly — on new, unseen data.
Classification Metrics
| Metric | What It Measures |
|---|---|
| Accuracy | Overall percentage of correct predictions |
| Precision | Of predicted positives, how many were actually positive |
| Recall | Of actual positives, how many were correctly identified |
| F1-Score | Harmonic mean of precision and recall |
| ROC-AUC | Model's ability to distinguish between classes across thresholds |
Regression Metrics
- MAE (Mean Absolute Error): Average absolute difference between predicted and actual values.
- MSE / RMSE: Penalizes larger errors more heavily than MAE.
- R-squared: Proportion of variance in the target explained by the model.
Validation Strategies
- Train/Test Split: A simple one-time split to estimate generalization.
- K-Fold Cross-Validation: Splits data into K folds, training and testing K times for a more robust estimate.
- Stratified Sampling: Preserves class proportions across folds, important for imbalanced data.
Why Accuracy Alone Can Mislead
On an imbalanced dataset (e.g., 95% non-fraud, 5% fraud), a model that always predicts "non-fraud" would score 95% accuracy while being completely useless. This is why precision, recall, and F1-score matter — especially for imbalanced problems.
Key Takeaway: Choose evaluation metrics that reflect the actual cost of errors in your business problem — accuracy alone rarely tells the full story.
Ready to master Machine Learning?
Build real-world machine learning skills with hands-on projects, mentor-led sessions, and industry-relevant tools.