Machine Learning Rapid-Fire Questions &amp; Answers

Show answer

Logistic regression computes a weighted sum of features and passes it through the sigmoid function, 1 / (1 + e^-z), to get a probability between 0 and 1. It is trained by minimising log loss. Coefficients represent change in log-odds per unit of a feature, and exp(coefficient) gives the odds ratio. A threshold, 0.5 by default, converts probabilities into class labels.

What are learning curves and validation curves?

Show answer

A learning curve plots training and validation scores against training set size; if both converge at a low score the model underfits, and if a gap remains, more data may help. A validation curve plots scores against one hyperparameter, such as max_depth, showing where overfitting starts. Both help decide whether to collect data, add features or change complexity.

What is the difference between offline evaluation and online A/B testing of a model?

Show answer

Offline evaluation measures metrics like AUC on historical holdout data; it is fast and cheap but cannot capture user behaviour changes or feedback loops. Online A/B testing exposes a share of real traffic to the new model and measures business outcomes like conversion or loss rate. Offline gains do not always translate online, so important models are validated with A/B tests or shadow deployments.

Why is feature scaling needed and which algorithms need it?

Show answer

Scaling puts features on comparable ranges so that one large-valued feature does not dominate distances or gradients. Distance and gradient based models need it: k-NN, k-means, SVM, PCA, linear and logistic regression with regularisation, and neural networks. Tree-based models like random forest and XGBoost split on thresholds, so they are largely unaffected by scaling.

What is the difference between Ridge, Lasso and Elastic Net regression?

Regression ModelsMedium

Show answer

All add a penalty on coefficient size to reduce overfitting. Ridge uses an L2 penalty (sum of squares), shrinking coefficients toward zero but rarely to zero, and handles multicollinearity well. Lasso uses L1 (sum of absolute values) and can set coefficients exactly to zero, performing feature selection. Elastic Net mixes both, which is useful when there are groups of correlated features.

Explain precision, recall and F1 score.

Clustering and Dimensionality ReductionEasy

Show answer

Precision is TP / (TP + FP): of the cases flagged positive, how many were right. Recall is TP / (TP + FN): of the actual positives, how many were found. F1 is the harmonic mean of the two and is low if either is low. Use precision when false alarms are costly, such as spam filters; use recall when misses are costly, such as cancer screening or fraud.

How do you choose the number of clusters k?

Show answer

Common methods are the elbow method, plotting inertia against k and looking for the bend; the silhouette score, which measures how close points are to their own cluster versus the next one; and the gap statistic. Business usability matters too: four clear segments a marketing team can act on are better than nine statistically optimal but confusing ones.

Explain hierarchical clustering and dendrograms.

Show answer

Agglomerative hierarchical clustering starts with each point as its own cluster and repeatedly merges the two closest clusters. Linkage defines closeness: single, complete, average or Ward, which minimises variance. A dendrogram shows the merge history, and cutting it at a height gives the clusters. It does not need k upfront but scales poorly, roughly O(n squared) memory, so it suits smaller datasets.

What is PCA and how does it work?

Show answer

Principal Component Analysis finds new orthogonal axes, the principal components, that capture maximum variance in the data. It works by computing eigenvectors of the covariance matrix, or via SVD, after centring and usually standardising the data. Keeping the top components that explain, say, 95 percent of variance reduces dimensions, removes correlation and speeds up models, but the components are harder to interpret.

What is the softmax function?

Classification ModelsMedium

Show answer

Softmax turns a vector of raw scores into probabilities that sum to one: p_i = exp(z_i) / sum of exp(z_j). It is used in multinomial logistic regression and the output layer of neural networks for multiclass problems. Larger scores get exponentially larger shares. In practice the maximum score is subtracted before exponentiation to avoid numerical overflow.

What is k-fold cross-validation and why is it better than a single split?

Model Evaluation and TuningEasy

Show answer

k-fold cross-validation splits training data into k folds, trains on k-1 folds and validates on the remaining one, repeating k times so every row is validated once. Averaging the k scores gives a more stable estimate than one random split and shows variance across folds. Five or ten folds are common. Use StratifiedKFold for classification and GroupKFold when rows from one user must stay together.

What is the difference between Gini impurity and entropy?

Decision Trees and EnsemblesEasy

Show answer

Gini impurity is 1 - sum of p squared and measures the chance of misclassifying a random sample. Entropy is - sum of p log p and comes from information theory; the reduction is called information gain. Both usually produce very similar trees. Gini is slightly faster because it avoids logarithms, which is why it is the scikit-learn default.

Explain the k-nearest neighbours algorithm and its weaknesses.

Show answer

k-NN predicts by finding the k closest training points and taking a majority vote for classification or the average for regression. It has no training phase but is slow at prediction on large data, needs feature scaling, suffers from the curse of dimensionality and is sensitive to irrelevant features. Small k overfits, large k underfits; choose k with cross-validation.

How do you detect and treat outliers?

Data Preprocessing and Feature EngineeringMedium

Show answer

Detect with box plots, the IQR rule (below Q1 - 1.5 IQR or above Q3 + 1.5 IQR), z-scores above 3, or models like Isolation Forest. Treatment depends on cause: fix data entry errors, cap values (winsorise), apply log transforms for skewed data, or use robust models and losses. Genuine extreme values, such as large fraud amounts, may be the most important signal and should not be removed blindly.

When would you use Huber or quantile regression instead of OLS?

Regression ModelsHard

Show answer

Huber regression uses squared loss for small errors and absolute loss for large ones, so it resists outliers while staying smooth. Quantile regression predicts a chosen percentile, such as the 90th percentile of delivery time, which is useful for setting safe promises or prediction intervals. Gradient boosting libraries support both losses for non-linear versions.

How does ordinary least squares find the regression coefficients?

Regression ModelsMedium

Show answer

OLS chooses coefficients that minimise the sum of squared residuals between actual and predicted values. It has a closed-form solution, beta = (X^T X)^-1 X^T y, when X^T X is invertible. For very large or ill-conditioned data, gradient descent or a pseudo-inverse is used instead. Squaring errors penalises large mistakes heavily, which makes OLS sensitive to outliers.

What is the difference between a regression and a classification problem?

Show answer

Regression predicts a continuous number, such as sales next month or delivery time in minutes. Classification predicts a discrete category, such as fraud or not fraud, or one of five product types. The choice decides the loss function and metrics: regression uses MSE, MAE or R-squared, while classification uses log loss, accuracy, precision, recall or AUC.

What is polynomial regression and what risk does it carry?

Regression ModelsMedium

Show answer

Polynomial regression adds powers and interaction terms of features, such as x squared and x1 times x2, so a linear model can fit curves. It is still linear in its coefficients. The main risk is overfitting at high degrees, plus wild behaviour outside the training range. Use cross-validation to choose the degree, scale features, and combine it with Ridge regularisation.

Compare grid search, random search and Bayesian optimisation.

Show answer

Grid search tries every combination of listed values; it is thorough but expensive and wastes trials on unimportant parameters. Random search samples combinations and usually finds good settings faster in high-dimensional spaces. Bayesian optimisation, as in Optuna or Hyperopt, builds a model of past trials to pick promising settings next, using fewer trials. Successive halving methods add early pruning of weak trials.

What is machine learning and how is it different from traditional programming?

Show answer

Machine learning is a way of building programs that learn patterns from data instead of following hand-written rules. In traditional programming you give rules and data to get answers; in ML you give data and answers to get the rules, stored as model parameters. For example, a spam filter learns which word patterns indicate spam from labelled emails rather than from a fixed keyword list.

What does it mean to deploy a machine learning model?

MLOps and DeploymentEasy

Show answer

Deployment makes a trained model available to produce predictions for real users or systems. It can be batch, scoring data on a schedule and writing results to a table, or real-time, serving predictions through a REST API, or streaming, scoring events as they arrive. It includes packaging the model with its preprocessing, setting up infrastructure, logging, monitoring and a retraining plan.

What is the log loss metric and why is it useful?

Clustering and Dimensionality ReductionEasy

Show answer

Log loss, or cross-entropy, is the negative average log of the probability assigned to the true class. It heavily penalises confident wrong predictions, such as predicting 0.99 for an event that does not happen. It evaluates probability quality, not just ranking, so it suits problems where probabilities drive decisions. Lower is better, and a constant base-rate prediction is the baseline to beat.

How does a decision tree decide where to split?

Decision Trees and EnsemblesEasy

Show answer

At each node the tree tries candidate features and thresholds and picks the split that most reduces impurity in the child nodes. For classification, impurity is measured by Gini or entropy; for regression, by variance or MSE. The process repeats recursively until a stopping rule such as max_depth, min_samples_leaf or pure leaves is reached. This greedy approach is fast but not globally optimal.

What is the difference between label encoding and ordinal encoding?

Data Preprocessing and Feature EngineeringMedium

Show answer

Both map categories to integers. In scikit-learn, LabelEncoder is meant for the target variable, while OrdinalEncoder is for input features. Integer encoding implies an order, so it is fine for truly ordered values like low, medium, high, or for tree models, but it can mislead linear models on nominal data like colours. For nominal features with linear models, prefer one-hot encoding.

Why containerise an ML model with Docker?

MLOps and DeploymentEasy

Show answer

Docker packages the model, code, Python version and exact library versions into one image that runs the same on a laptop, a test server or Kubernetes. This removes 'works on my machine' problems, especially with scikit-learn version mismatches. Images can be scanned, versioned and deployed through CI/CD, and scaled horizontally behind a load balancer on platforms like Kubernetes, ECS or Cloud Run.

How does k-means clustering work?

Show answer

k-means picks k initial centroids, assigns each point to its nearest centroid, recomputes each centroid as the mean of its points, and repeats until assignments stop changing. It minimises within-cluster sum of squares (inertia). It needs k in advance, scaled features, and assumes roughly spherical, similar-sized clusters. k-means++ initialisation and multiple runs with n_init reduce bad starting points.

What is association rule mining? Explain support, confidence and lift.

Show answer

Association rule mining finds items that occur together, such as in market basket analysis. Support is how often an itemset appears in all transactions. Confidence of A to B is the share of transactions with A that also contain B. Lift is confidence divided by B's support; lift above 1 means A and B occur together more than by chance. Apriori and FP-Growth are common algorithms.

What is R-squared and what is adjusted R-squared?

Regression ModelsEasy

Show answer

R-squared is the share of variance in the target explained by the model, 1 - SS_res / SS_tot. It never decreases when you add features, even useless ones. Adjusted R-squared penalises extra features: 1 - (1 - R2)(n - 1)/(n - p - 1), so it only rises if a new feature improves fit more than expected by chance. R-squared can be negative on test data if the model is worse than predicting the mean.

Compare MAE, MSE and RMSE as regression metrics.

Regression ModelsEasy

Show answer

MAE is the average absolute error, easy to explain and robust to outliers. MSE averages squared errors and punishes large errors heavily, which suits cases where big misses are costly. RMSE is the square root of MSE, so it is in the same unit as the target. If RMSE is much higher than MAE, a few large errors are present. MAPE is useful for percentage errors but fails near zero.

What are the assumptions of linear regression?

Regression ModelsEasy

Show answer

Linear regression assumes a linear relationship between features and target, independent errors, constant error variance (homoscedasticity), normally distributed errors for valid inference, and little multicollinearity among features. Violations mainly hurt confidence intervals and coefficient interpretation more than raw predictions. Check them with residual plots, Q-Q plots, the Durbin-Watson test and VIF.

What is the difference between standardisation and normalisation?

Show answer

Standardisation (StandardScaler) transforms a feature to mean 0 and standard deviation 1 using (x - mean) / std; it keeps outliers but does not bound values. Min-max normalisation (MinMaxScaler) rescales to a fixed range such as 0 to 1 and is sensitive to outliers. RobustScaler uses median and IQR and is better when outliers are present. Always fit the scaler on training data only.

Explain supervised, unsupervised and reinforcement learning with one example each.

Show answer

Supervised learning trains on labelled data, such as predicting house prices from past sales. Unsupervised learning finds structure in unlabelled data, such as grouping customers into segments with k-means. Reinforcement learning trains an agent through rewards and penalties from an environment, such as a program learning to play a game. Semi-supervised and self-supervised learning sit between these when labels are scarce.

What is multicollinearity and how do you detect it?

Data Preprocessing and Feature EngineeringMedium

Show answer

Multicollinearity is high correlation between input features, which makes linear model coefficients unstable and hard to interpret, though predictions may remain fine. Detect it with a correlation matrix or the Variance Inflation Factor, where VIF above 5 to 10 signals a problem. Fix it by dropping one of the correlated features, combining them, using PCA, or applying Ridge regularisation.

What is the difference between batch, stochastic and mini-batch gradient descent?

ML FundamentalsMedium

Show answer

Batch gradient descent uses the full dataset for each update, giving stable but slow and memory-heavy steps. Stochastic gradient descent updates after every single example, which is fast and noisy and can escape shallow minima. Mini-batch uses small groups such as 32 to 512 rows, balancing stability with speed and allowing vectorised computation. Mini-batch is the default in practice.

How do you handle class imbalance?

Classification ModelsMedium

Show answer

Start with the right metrics, such as PR-AUC, recall at fixed precision or F1, not accuracy. Then use class_weight='balanced', resampling like random undersampling or SMOTE applied only on training folds, and threshold tuning based on business cost. Boosting models with scale_pos_weight also help. Stratified splits keep class ratios consistent across folds.

What is the ROC curve and AUC? When is PR-AUC better?

Classification ModelsMedium

Show answer

The ROC curve plots true positive rate against false positive rate at every threshold, and AUC is the probability that a random positive is ranked above a random negative. It is threshold-independent. On heavily imbalanced data, like 0.5 percent fraud, ROC-AUC can look high even when precision is poor, so the precision-recall curve and PR-AUC give a more honest picture.

What is feature engineering and why does it matter?

Show answer

Feature engineering is creating input variables that make patterns easier for a model to learn, using domain knowledge. Examples are extracting day of week from a date, ratios like debt-to-income, rolling averages of past purchases, or text length. On tabular problems good features often improve results more than switching algorithms. Features must use only information available at prediction time.

What is the bias-variance trade-off?

ML FundamentalsMedium

Show answer

Bias is error from overly simple assumptions, which causes underfitting. Variance is error from sensitivity to the particular training sample, which causes overfitting. Increasing model complexity usually lowers bias but raises variance. The goal is the complexity that minimises total test error. For example, a linear model on curved data has high bias, while a very deep unpruned tree has high variance.

Why do single decision trees overfit and how do you control it?

Decision Trees and EnsemblesEasy

Show answer

An unrestricted tree keeps splitting until leaves are pure, memorising noise. Control it with pre-pruning hyperparameters like max_depth, min_samples_split, min_samples_leaf and max_leaf_nodes, or with post-pruning through cost-complexity pruning using ccp_alpha. In practice ensembles such as random forests or boosting are used to reduce the variance of single trees.

What is a loss function? Name common ones for regression and classification.

ML FundamentalsMedium

Show answer

A loss function measures how wrong a prediction is for one example, and training minimises its average over the data. Regression commonly uses mean squared error, mean absolute error or Huber loss, which is robust to outliers. Classification uses log loss, also called cross-entropy, and hinge loss for SVMs. The choice should reflect the business cost of errors.

What is a confusion matrix?

Show answer

A confusion matrix is a table of actual versus predicted classes. For binary problems it has true positives, false positives, true negatives and false negatives. From it you derive precision, recall, specificity and accuracy. For example, in fraud detection a false negative is a missed fraud and a false positive is a genuine customer blocked, and these usually have very different costs.

What is DBSCAN and when is it better than k-means?

Show answer

DBSCAN groups points that lie in dense regions, defined by eps (neighbourhood radius) and min_samples, and labels points in sparse areas as noise. It finds arbitrarily shaped clusters, does not need k, and handles outliers. It struggles when clusters have very different densities and in high dimensions. HDBSCAN improves on it for varying densities.

Why do we split data into training, validation and test sets?

Show answer

The training set fits model parameters, the validation set is used to choose hyperparameters and compare models, and the test set gives a final unbiased estimate of performance. If you tune on the test set, its score becomes optimistic because information has leaked into model choices. A common split is 70/15/15, or train/test plus cross-validation on the training part.

What makes XGBoost, LightGBM and CatBoost different?

Decision Trees and EnsemblesMedium

Show answer

XGBoost adds regularisation to the objective, uses second-order gradients and handles missing values natively. LightGBM grows trees leaf-wise and uses histogram binning, making it very fast on large data, though it can overfit small data. CatBoost handles categorical features with ordered target statistics and uses symmetric trees, giving strong defaults with less tuning. All three support GPU training.

What is MLOps?

MLOps and DeploymentEasy

Show answer

MLOps applies DevOps practices to machine learning: version control for code, data and models, automated training pipelines, testing, CI/CD for model releases, model registries, monitoring and automated retraining. Its aim is to move models from notebooks to production reliably and keep them healthy. Common tools are MLflow, DVC, Airflow, Kubeflow, and cloud platforms like SageMaker, Vertex AI and Azure ML.

How does Isolation Forest detect anomalies?

Show answer

Isolation Forest builds random trees that split on random features and random thresholds. Anomalies are few and different, so they get isolated in fewer splits, giving shorter average path lengths. The anomaly score comes from that path length. It is fast, works without labels and scales well. The contamination parameter sets the expected share of anomalies and determines the threshold.

How do you handle missing values in a dataset?

Show answer

First understand why values are missing: completely at random, at random, or not at random. Options are dropping rows or columns with heavy missingness, simple imputation with mean, median or mode, model-based imputation like KNNImputer or IterativeImputer, or a separate 'missing' category. Adding a missing-indicator column often helps because missingness itself can be predictive, such as income left blank.

What is Naive Bayes and why is it called naive?