Machine Learning Last-Minute Revision

30 quick revision notes for Machine Learning

1. Supervised learning uses labelled data, unsupervised finds structure without labels, and reinforcement learning learns from rewards in an environment.
2. High bias causes underfitting and high variance causes overfitting; compare training and validation scores to tell which one you have.
3. Fit scalers, imputers and encoders on training data only, ideally inside a scikit-learn Pipeline, to avoid data leakage.
4. Tree-based models do not need feature scaling, but k-NN, SVM, PCA, k-means and regularised linear models do.
5. One-hot encoding suits low-cardinality nominal features; use frequency or out-of-fold target encoding for high-cardinality columns.
6. Ridge (L2) shrinks coefficients smoothly, Lasso (L1) can set them exactly to zero, and Elastic Net combines both penalties.
7. R-squared never falls when features are added; adjusted R-squared penalises extra features that do not help.
8. RMSE punishes large errors more than MAE; a big gap between RMSE and MAE signals a few large misses.
9. Logistic regression models log-odds with a sigmoid and is trained by minimising log loss; exp of a coefficient is an odds ratio.
10. Precision is TP over TP plus FP, recall is TP over TP plus FN, and F1 is their harmonic mean.
11. On heavily imbalanced data, accuracy and even ROC-AUC can mislead; prefer PR-AUC, recall at fixed precision or cost-based metrics.
12. The default 0.5 threshold is rarely optimal; tune it on validation data based on business cost or review capacity.
13. Apply SMOTE or any resampling only inside training folds, never before the train-test split.
14. Calibrated models output probabilities that match observed frequencies; check with reliability curves and the Brier score.
15. Random forests use bagging plus random feature subsets to reduce variance; out-of-bag samples give a free validation estimate.
16. Gradient boosting fits each new tree to the negative gradient of the loss and needs a learning rate plus early stopping.
17. LightGBM grows trees leaf-wise with histogram binning, XGBoost adds regularisation, and CatBoost handles categorical features natively.
18. Impurity-based feature importance is biased toward high-cardinality features; permutation importance and SHAP are more reliable.
19. k-means needs k in advance and scaled features; choose k with the elbow method, silhouette score and business usefulness.
20. DBSCAN finds arbitrary-shaped clusters and marks noise points using eps and min_samples, without needing the number of clusters.
21. PCA finds orthogonal directions of maximum variance; standardise data first and keep components that explain about 95 percent variance.
22. t-SNE and UMAP are for visualising local structure; distances between t-SNE clusters should not be interpreted.
23. Use StratifiedKFold for classification, GroupKFold for repeated entities and TimeSeriesSplit for time-ordered data.
24. Random search and Bayesian optimisation with Optuna usually find good hyperparameters faster than exhaustive grid search.
25. The test set should be used once for the final estimate; repeated tuning on it makes the score optimistic.
26. Data drift is a change in input distribution; concept drift is a change in the relationship between inputs and target.
27. PSI below 0.1 means stable, 0.1 to 0.25 moderate shift, and above 0.25 a significant drift that needs investigation.
28. Training-serving skew happens when features are computed differently in production; share one feature code path or a feature store.
29. Save the full fitted pipeline with joblib, pin library versions and never load pickle files from untrusted sources.
30. Shadow deployments compare a new model on live traffic without affecting users; canary releases expose it to a small share first.

Night-before checklist

• Revise your saved questions on this page once, aloud.
• Read the quick notes and mark any you cannot explain in one line.
• Rehearse a two-minute self-introduction and two project stories.
• Keep CV, ID, laptop charger and interview link ready; sleep on time.