Scenario-Based Machine Learning Interview Questions

This is overfitting. I would first check for leakage or a split mismatch, then simplify the model: add regularisation, reduce depth or features, use early stopping, or prune trees. I would gather more data or use augmentation if possible and use cross-validation to confirm the fix. Plotting learning curves shows whether more data would actually help.

I would explain that for small tabular data, gradient-boosted trees or regularised linear models usually match or beat neural networks, train faster and are easier to explain. I would propose building a simple baseline, then a boosting model, and only test a neural network if there is a clear reason. I would show the comparison on cross-validated metrics so the decision is data-driven.

First define churn precisely, for example no purchase in 60 days, and the prediction window. It becomes binary classification. Build features from data available before the cut-off date only, choose a time-based split, and pick metrics like recall at a fixed precision or lift in the top decile. Start with logistic regression as a baseline, then try boosting, and agree on how retention teams will use the scores.

Common causes are a learning rate that is too high or too low, unscaled features, a bug in labels or feature alignment, wrong loss for the task, or data that has been shuffled away from its labels. I would try overfitting a tiny subset first; if the model cannot even fit 50 rows, there is a bug. Then check scaling, learning rate and gradient values.

I would agree on a narrow first version: a simple, explainable baseline built on the cleanest available fields, with clear notes on data gaps and expected accuracy. In parallel I would log specific data issues and their impact. Delivering a working baseline with an honest limitations section builds trust and gives a reference point for improvement in later iterations.

One-hot would create 2,000 sparse columns, so I would first group rare cities below a frequency threshold into 'Other'. Then options are frequency encoding, out-of-fold target encoding, or mapping cities to useful attributes such as tier, state or population. For boosting models like CatBoost or LightGBM, native categorical handling also works well. I would compare options with cross-validation.

Such a high score on a hard problem usually signals leakage. I would check feature importances for a suspiciously dominant feature, look for fields recorded after the loan outcome such as collection status, verify preprocessing was fit only on training data, and check for duplicate customers across train and test. Then re-run with a time-based split to confirm a realistic score.

I would study the missingness pattern first, for example whether self-employed applicants skip it. I would add a binary was_missing feature, impute with the median within relevant groups or use IterativeImputer, and compare against models that handle missing values natively, like HistGradientBoosting or XGBoost. I would also raise the issue with the data owner so collection can improve.

Make the encoder tolerant: OneHotEncoder with handle_unknown='ignore' or infrequent_if_exist, or map unseen values to an 'Other' bucket. Monitor the share of unseen categories as a drift signal. If new categories become common, such as a new city launch, retrain with fresh data. Tests in the inference pipeline should include unseen values so the service never crashes.

That is heteroscedasticity, typical for price data that is right-skewed. I would model log(price) instead of price, which turns absolute errors into roughly percentage errors and stabilises variance. Then I would convert predictions back with exp and check residual plots again. Evaluating with RMSLE or MAPE also fits better when relative error matters more than absolute error.

Lasso often keeps only one of a group of correlated features, so the dropped feature may be redundant rather than useless. I would check correlations, try Elastic Net which keeps correlated groups, and compare cross-validated performance with the feature forced in. I would explain the trade-off clearly; if the accuracy difference is small, keeping the feature for trust and interpretability is reasonable.

The model lacks festival information. I would add features for festival dates such as Diwali or regional holidays, days-to-festival, and past festival uplift for the same item. If the business cost of stock-outs is higher than over-stock, I would use a quantile loss for a higher percentile. Then validate specifically on past festival periods, not just the average error.

It depends on error cost. The first model makes fewer big errors, while the second is better on typical cases but has some large misses. If large errors are expensive, such as stock-outs of costly items, choose the first. If typical accuracy matters more, choose the second. I would also check error distributions by segment and discuss with stakeholders before deciding.

Options are quantile regression, training separate models for the 10th and 90th percentiles with GradientBoostingRegressor(loss='quantile') or LightGBM; conformal prediction, which wraps any model and gives intervals with guaranteed coverage; or bootstrapping. I would check calibration: about 80 percent of actual values should fall within an 80 percent interval on a holdout set.

If only 0.4 percent of transactions are fraud, predicting 'not fraud' for everything gives 99.6 percent accuracy and catches nothing. I would switch to recall, precision and PR-AUC, check the confusion matrix, then retrain with class weights or resampling and tune the threshold so the fraud team gets a manageable number of alerts with high recall.

Instead of 0.5, rank transactions by predicted probability and pick the threshold that yields about 500 alerts per day on recent data. Then report precision and recall at that cut-off, or precision at k. I would monitor daily alert volume because drift can change it, and recompute the threshold periodically. If cost data exists, I would choose the threshold that maximises expected savings.

I would measure metrics per segment to confirm, then check if that segment is under-represented or has different feature distributions. Fixes include collecting more data, sample weighting, segment-specific features, or a separate model if the behaviour is truly different. If the segment relates to a protected attribute, I would also review fairness metrics and involve the compliance team.

The model is poorly calibrated: it ranks customers but its probabilities are not reliable. Since pricing depends on actual probabilities, I would calibrate it with isotonic regression or Platt scaling on a held-out set and validate with a reliability curve and Brier score. I would also check whether the default rate has drifted since training, which would require retraining.

I would use TF-IDF features with a Multinomial Naive Bayes or linear model like logistic regression or LinearSVC. They train in seconds, handle high-dimensional sparse text well and give strong baselines. I would evaluate precision carefully because blocking genuine mail is costly. Only if this is not good enough would I try fine-tuning a small transformer model.

Reduce size by limiting max_depth and min_samples_leaf, using fewer trees since gains flatten after a few hundred, and dropping weak features. Alternatively switch to a gradient boosting model like LightGBM, which usually gives similar accuracy with far smaller, faster models. Converting to ONNX or using a compiled predictor can also speed up inference. Measure accuracy and latency after each change.

That is overfitting beyond about 300 rounds. I would use early stopping with a patience such as 50 rounds on a validation set so training stops at the best iteration, then lower the learning rate, reduce max_depth, add subsample and colsample_bytree below 1, and increase min_child_weight or lambda. The final model is refit using the best iteration count found.

I would use SHAP to show global drivers with a summary plot and explain individual decisions with waterfall plots, such as why one applicant got a high risk score. I would also check that the top drivers make business sense, add monotonic constraints where logic requires it, and share a few real case examples. If trust is still an issue, a scorecard model can serve as a transparent benchmark.

First check whether data supports it, perhaps due to confounding or a small segment. If the relationship should be one-directional for business or regulatory reasons, I would apply monotonic constraints, available in XGBoost, LightGBM and HistGradientBoosting, so risk can only decrease with income. Then compare validation metrics; usually the loss is small and the model becomes more trustworthy.

I would compare beyond AUC: model size, inference latency, training time, calibration and stability across time periods. LightGBM is usually smaller and faster to score, while random forest needs less tuning and is less prone to overfitting. If both are equal, I would choose the one that is cheaper to run and easier for the team to maintain and retrain.

I would agree on the purpose first, such as retention or upsell. Then build RFM features (recency, frequency, monetary) plus category preferences, log-transform skewed values and scale them. I would try k-means for k between 3 and 8, use silhouette and business review to pick k, profile each segment and give them clear names. Finally, test segment-based campaigns against a control group.

Likely causes are unscaled features, where one large-range feature dominates distance; heavy skew or outliers pulling centroids; or data that is not naturally spherical. I would log-transform and scale features, remove or cap extreme outliers, and try different k values. If shapes are irregular, DBSCAN, HDBSCAN or a Gaussian Mixture Model may fit better.

I would start with unsupervised anomaly detection: Isolation Forest or Local Outlier Factor on features like amount relative to the customer's usual spend, time of day, new device and merchant risk. I would set the threshold based on what the review team can handle, collect investigator feedback as labels, and move to a supervised model once enough labelled cases exist.

I would remove constant and near-duplicate features, drop one from each highly correlated pair, then apply PCA inside the pipeline and keep components explaining about 95 percent of variance. Alternatively, use L1-regularised models or tree importances for supervised selection. I would compare speed and cross-validated scores; often 50 to 100 features give nearly the same accuracy.

I would measure drift by comparing feature distributions and segment sizes over time. Then refit clustering on recent data and map new clusters to old ones by centroid similarity so business names stay consistent. I would set up a quarterly refresh with stability checks and alert when segment sizes shift sharply, and explain any renamed or merged segments to stakeholders.

Common causes are preprocessing written separately in two places, statistics like means recomputed on production data instead of using training values, different library versions, or different column order and data types. The fix is to save one fitted Pipeline with preprocessing and model together, pin versions, validate input schema, and test that the saved pipeline gives identical outputs on sample rows.

Possible causes: leakage inside CV, such as preprocessing or feature selection fit outside folds; repeated tuning that overfit the CV folds; a test set from a different time period or population; or duplicate entities across folds but not test. I would check pipeline structure, switch to time-based or group splits that match how test data differs, and compare feature distributions between the two sets.

I would pick the model that does better where decisions are actually made. If the team only reviews the top 1,000 cases, precision at 1,000 or recall at that cut-off matters more than global AUC. I would make that the primary metric in future comparisons, and confirm the result holds across several time periods before switching.

Switch to random search or Optuna with pruning, which find good settings with far fewer trials. Narrow ranges using a quick coarse search, tune the most important parameters first, use early stopping for boosting, run trials in parallel with n_jobs, and tune on a stratified sample of the data before a final refit on the full set. HalvingRandomSearchCV is another quick option.

I would convert model outcomes into money or time using a cost matrix agreed with the business, for example savings from caught fraud minus the cost of reviewing false alerts. I would compare this against the current process or a rule-based baseline on holdout data and show a range rather than one point. A short note on assumptions keeps the number honest.

I would check the test first: enough sample size and duration, correct randomisation, and no bugs in serving such as stale features. Then investigate whether offline metrics really reflect user value, for example optimising clicks rather than purchases, or popularity bias. Segment-level analysis may show gains in some groups. I would revise the offline metric to better match the online goal.

I would first check data pipelines for broken features, nulls or schema changes, then compare current feature distributions with training data using PSI to find drift. I would check whether the label definition or business process changed, and analyse errors by segment. If drift is real, retrain on recent data, possibly with time-weighted samples, and set up automatic drift alerts and scheduled retraining.

Profile first to see whether time goes to feature lookup, preprocessing or the model. Then cache or precompute features, load the model once at startup, reduce trees or features, use a faster library like LightGBM or ONNX Runtime, and batch requests if possible. Run several workers and keep the service close to its data store. Re-measure p95 and p99 latency, not just the average.

Roll back immediately to the previous version from the model registry, which should be a one-step action. Then investigate: compare the input features served with those used in validation, check for training-serving skew, version mismatches or a wrong artifact. Add the failure case to automated pre-release checks, and use canary or shadow releases next time so issues affect only a small share of traffic.

I would rely on proxy signals: input data drift with PSI, prediction score distribution, approval rates by segment, and early indicators like first missed payment at 30 or 60 days. Once enough early outcomes arrive, I would check performance on these. I would also set up a delayed evaluation job that computes full metrics as labels mature and compares them with the validation baseline.

I would convert the notebook into a versioned training pipeline with separate steps for data extraction, validation, feature building, training and evaluation, scheduled with Airflow or a cloud pipeline. Each run would log to MLflow, compare against the production model on a fixed holdout, and register the new model only if it passes thresholds. Promotion would stay a reviewed step, at least initially.

Model size, memory and latency on low-end phones become key. I would choose a compact model, reduce features, and convert it to a mobile format such as ONNX, TensorFlow Lite or Core ML, applying quantisation if needed. I would make sure all preprocessing is replicated on-device, test accuracy after conversion, and plan how model updates will ship with app releases.

I would check for drift between periods in feature distributions and target rate, and confirm that the evaluation uses a time-based split. If behaviour has changed, I would train on a more recent window or weight recent data more, add features that capture the change, and set up a regular retraining schedule. Comparing SHAP importance across periods shows which relationships have shifted.

I would build product embeddings from content: TF-IDF or sentence embeddings of titles and descriptions, plus category and price features. Then use cosine similarity with nearest-neighbour search, such as scikit-learn NearestNeighbors or an approximate index like FAISS for large catalogues. I would review samples with the product team and later add co-view or co-purchase signals once collected.