01
How do you tune hyperparameters?
Show answer
Use grid search, random search or Bayesian optimisation with cross-validation, starting with a coarse random search and narrowing the promising ranges.
Medium Data Science Interview Questions
209 medium questions with answers. Applied questions that check whether you can use the concepts on real work.
Topics in the medium set
Machine Learning: 36 Deep Learning and NLP: 28 Python for Data Science: 21 Statistics and Probability: 19 Feature Engineering: 17 SQL for Data Science: 17 MLOps and Model Deployment: 16 Generative AI and LLMs: 16 Computer Vision: 14
Solved — mediumStart practicing
3 QUESTIONS FREESign up to unlock the full question bank and track your progress.Unlock every question and answer, then continue practising company-wise and package-wise.
3 questions free Unlock the rest in seconds
02
How does the Gini impurity differ from entropy?
Show answer
Both measure node impurity; Gini is cheaper to compute and entropy is slightly more sensitive to class balance, with similar trees in practice.
03
What is the difference between an epoch, a batch and an iteration?
Show answer
An epoch is one pass over the training data; a batch is the subset used for one update; an iteration is one weight update on a batch.
04
What is statistical power and how can you increase it?
Show answer
Power is the probability of detecting a real effect. Increase it with a larger sample, a bigger effect size, lower noise or a higher significance level.
05
What is calibration of a classifier?
Show answer
Calibration checks whether predicted probabilities match real frequencies (e.g. 70% predictions are right about 70% of the time). Fix with Platt scaling or isotonic regression.
06
Why do we use activation functions? Why is ReLU popular?
Show answer
Activations add non-linearity so networks can learn complex patterns. ReLU (max(0, x)) is cheap and reduces vanishing gradients compared with sigmoid/tanh; Leaky ReLU avoids 'dead' neurons.
07
What is the vanishing gradient problem?
Show answer
Gradients shrink as they pass through many layers (for example with sigmoid), so early layers barely learn. ReLU, residual connections and good initialisation help.
08
Which activation functions do you know and when to use them?
Show answer
ReLU for hidden layers, sigmoid for binary outputs, softmax for multi-class outputs and tanh when zero-centred outputs help; Leaky ReLU avoids dead neurons.
09
How do you perform feature selection?
Show answer
Use filter methods (correlation, chi-square), wrapper methods (RFE), embedded methods (Lasso, tree importance) and domain knowledge, validating each choice with cross-validation.
10
What is feature scaling and which models need it?
Show answer
Putting features on comparable ranges using standardisation or min-max scaling. Distance-based and gradient-based models (KNN, SVM, regression, neural nets) need it; tree models do not.
11
Explain the Law of Large Numbers.
Show answer
As the number of independent trials grows, the sample average converges to the true expected value, which is why larger samples give more stable estimates.
12
What are RNNs and why did LSTMs improve them?
Show answer
RNNs process sequences with a hidden state but struggle with long dependencies. LSTMs add gates that control what to remember or forget, easing the vanishing gradient.
13
L1 vs L2 regularisation?
Show answer
L1 (Lasso) adds absolute weights and can shrink some to zero, giving feature selection. L2 (Ridge) adds squared weights and shrinks them smoothly without removing features.
14
Your dataset is highly imbalanced (2% positives). How do you handle it?
Show answer
Use stratified splits, class weights, resampling (SMOTE or undersampling) and threshold tuning, and judge the model with PR-AUC or F1 instead of accuracy.
15
What is the difference between parametric and non-parametric tests?
Show answer
Parametric tests assume a distribution (usually normal) and use means; non-parametric tests such as Mann-Whitney use ranks and make fewer assumptions.
16
What is an autoencoder and what is it used for?
Show answer
A network that compresses inputs into a small latent code and reconstructs them; used for denoising, dimensionality reduction and anomaly detection.
17
Difference between a shallow copy and a deep copy?
Show answer
A shallow copy duplicates the outer object but shares nested objects. A deep copy (copy.deepcopy) recursively duplicates everything, so changes to nested items do not leak.
18
How does K-means clustering work and how do you choose k?
Show answer
It assigns points to the nearest centroid and updates centroids until stable. Choose k with the elbow method, silhouette score or business context.
19
Explain ROC curve and AUC.
Show answer
The ROC curve plots true positive rate against false positive rate across thresholds. AUC is the probability that a random positive is ranked above a random negative; 0.5 is chance.
20
How do you pivot data in Pandas?
Show answer
pd.pivot_table(df, index='region', columns='month', values='sales', aggfunc='sum') reshapes long data into a summary matrix; melt() does the reverse.
21
What are decorators in Python?
Show answer
A decorator wraps a function to add behaviour such as logging, timing or caching without changing its code. It is applied with the @name syntax above the function.
22
How do you detect outliers?
Show answer
Use the IQR rule (below Q1-1.5 IQR or above Q3+1.5 IQR), z-scores beyond 3, box plots or model-based methods such as Isolation Forest, then decide whether to cap, remove or keep them.
23
What is regularization? Explain L1 vs L2.
Show answer
Regularization adds a penalty on weights to reduce overfitting. L1 (Lasso) adds |w| and can push features to exactly zero. L2 (Ridge) adds w² and shrinks weights smoothly.
24
What is a p-value?
Show answer
The probability of seeing a result at least as extreme as the observed one if the null hypothesis were true. A small p-value is evidence against the null, not proof that the alternative is true.
25
Why are NumPy arrays faster than Python lists?
Show answer
They store same-type values in contiguous memory and run vectorised operations in compiled C, avoiding Python loop overhead and per-object type checks.
26
What is the Global Interpreter Lock (GIL)?
Show answer
A mutex in CPython that lets only one thread execute Python bytecode at a time. Use multiprocessing for CPU-bound work and threads or asyncio for I/O-bound work.
27
What are lambda functions and where are they useful?
Show answer
Small anonymous one-line functions, handy as arguments to sort, map, filter or Pandas apply.
28
Pandas: find the top 3 products by revenue in each region.
Show answer
(df.assign(rev=df.price * df.qty)
.groupby(['region', 'product'])['rev'].sum()
.reset_index()
.sort_values('rev', ascending=False)
.groupby('region').head(3))
29
Explain the KNN algorithm and its drawbacks.
Show answer
It predicts from the k nearest labelled points. It is simple but slow at prediction time, sensitive to scaling and irrelevant features, and weak in high dimensions.
30
Pandas: merge vs join vs concat?
Show answer
merge joins on columns, SQL-style (how='inner', 'left', ...). join joins on the index. concat stacks DataFrames along rows or columns.
31
What is Naive Bayes and why is it called naive?
Show answer
It applies Bayes' theorem assuming features are independent given the class. The assumption is rarely true but the model is fast and works well for text.
32
How do you handle imbalanced classes?
Show answer
Use stratified splits, class weights, resampling such as SMOTE, threshold tuning and judge models with precision, recall, F1 or PR-AUC instead of accuracy.
33
Explain logistic regression.
Show answer
It models the log-odds of a binary outcome as a linear function and passes it through a sigmoid to give probabilities, trained by maximising likelihood.
34
What is TF-IDF?
Show answer
Term Frequency × Inverse Document Frequency: a word scores high if it is frequent in one document but rare across all documents. A classic way to turn text into numeric features.
35
What is stacking in ensemble learning?
Show answer
Training a meta-model on the predictions of several base models, often using cross-validated predictions to avoid leakage.
36
What is batch normalisation?
Show answer
It normalises layer inputs per mini-batch and then rescales them, stabilising and speeding up training and allowing higher learning rates.
37
How do you use groupby with multiple aggregations?
Show answer
df.groupby('region').agg(total=('sales','sum'), avg=('sales','mean')) gives named aggregations per group in one pass.
38
What is backpropagation?
Show answer
It computes the gradient of the loss with respect to every weight by applying the chain rule backwards through the network, so gradient descent can update them.
39
What is gradient descent and what does the learning rate do?
Show answer
An optimisation method that updates weights in the opposite direction of the loss gradient. The learning rate sets the step size: too high and it diverges, too low and it is slow. Variants include SGD, mini-batch and Adam.
40
What is correlation and why does it not imply causation?
Show answer
Correlation measures linear association between variables (-1 to 1). A third factor, reverse causality or chance can create it, so causation needs controlled experiments.
41
What is transfer learning?
Show answer
Reusing a model pretrained on a large dataset and fine-tuning it on a smaller task-specific dataset, saving data, time and compute.
42
How do you profile and speed up slow Python code?
Show answer
Measure first with cProfile or timeit, then replace loops with NumPy or Pandas vectorised calls, use built-ins, caching and better data structures.
43
What is a confidence interval?
Show answer
A range built from sample data that would contain the true parameter in a stated percentage (for example 95%) of repeated samples. It is not the probability that the parameter lies inside this one interval.
44
How does a CNN work?
Show answer
Convolution filters slide over the input to detect local patterns, pooling reduces size, and deeper layers combine edges into shapes and objects. Used mainly for images.
45
What is the bias–variance trade-off?
Show answer
Bias is error from overly simple assumptions (underfitting). Variance is sensitivity to the training data (overfitting). More model complexity lowers bias but raises variance — aim for the lowest total error using cross-validation and regularization.
46
What is a generator and when would you use one?
Show answer
A function using yield that produces values lazily, one at a time. It saves memory when processing large files or streams because the full sequence is never held in RAM.
47
How do you deal with high-cardinality categorical features?
Show answer
Use target or frequency encoding with cross-validation, hashing, embeddings or grouping rare categories, instead of huge one-hot matrices.
48
What is PCA and when do you use it?
Show answer
It projects data onto orthogonal directions of maximum variance to reduce dimensions, remove correlation and speed up models, at some cost of interpretability.
49
What are common ways to evaluate NLP models?
Show answer
Accuracy, F1 and confusion matrices for classification, BLEU or ROUGE for generation and summarisation, perplexity for language models, plus human review.
50
What is the difference between iterators and iterables in Python?
Show answer
An iterable can produce an iterator (it has __iter__); an iterator has __next__ and remembers its position. A for-loop calls iter() on the iterable and then next() repeatedly.
51
Random forest vs a single decision tree?
Show answer
A forest averages many trees trained on bootstrapped rows and random feature subsets, which cuts variance and overfitting at the cost of interpretability.
52
Why is XGBoost so popular?
Show answer
It combines gradient boosting with regularisation, handling of missing values, parallel tree construction and speed optimisations, and it performs strongly on tabular data.
53
What is the difference between apply, map and applymap in Pandas?
Show answer
map works element-wise on a Series, apply works on a Series or along a DataFrame axis, and applymap (now DataFrame.map) applies a function to every cell.
54
What is sampling bias and how can you reduce it?
Show answer
Bias occurs when the sample does not represent the population. Reduce it with random or stratified sampling, higher response rates and by checking sample demographics against the population.
55
How does a decision tree decide where to split?
Show answer
It tries candidate splits and picks the one that most reduces impurity — Gini or entropy (information gain) for classification, variance / MSE for regression.
56
How do you detect and treat concept drift?
Show answer
Monitor input distributions and live metrics, compare to training baselines, and retrain or recalibrate on fresh data when performance drops.
57
How does linear regression work and what are its assumptions?
Show answer
It fits a line minimising squared error. Assumes linearity, independent errors, constant variance (homoscedasticity), normally distributed residuals and no strong multicollinearity.
58
What is the attention mechanism?
Show answer
It lets a model weigh which input positions matter most for each output, computing weighted combinations of values from query-key similarity. It is the core of transformers.
59
What is SVM and the kernel trick?
Show answer
SVM finds the maximum-margin separating hyperplane. Kernels such as RBF compute similarity in a higher-dimensional space implicitly, so non-linear boundaries are possible.
60
Explain the chi-square test.
Show answer
It checks whether observed category counts differ from expected counts, used for independence between two categorical variables and for goodness of fit.
61
Explain Bayes' theorem with an example.
Show answer
P(A given B) = P(B given A) x P(A) / P(B). In a medical test it converts test accuracy and disease prevalence into the real chance a positive result means disease.
62
What are precision and recall trade-offs in practice?
Show answer
Raising the threshold improves precision but lowers recall. Fraud detection leans to recall, while spam filters often prefer precision; choose by the cost of each error.
63
What is the Central Limit Theorem?
Show answer
The distribution of sample means approaches a normal distribution as the sample size grows (roughly 30+), whatever the population's shape. It lets us build confidence intervals and run hypothesis tests on means.
64
How do you choose a learning rate for a neural network?
Show answer
Start with a sensible default such as 1e-3 for Adam, run a learning-rate range test, and use schedulers or warm-up while watching the loss curve.
65
How does dropout work?
Show answer
During training it randomly switches off a fraction of neurons so the network cannot depend on any single unit, acting as a regulariser against overfitting.
66
You assign values to a filtered DataFrame and Pandas shows a SettingWithCopyWarning. What is going on and how do you fix it?
Show answer
The warning appears when you assign values to something that may be a copy of a slice instead of the original DataFrame, for example df[df.age > 30]['flag'] = 1. The original may not change. Fix it by using a single .loc call, like df.loc[df.age > 30, 'flag'] = 1, or by taking an explicit .copy() when you really want a separate DataFrame. Copy-on-write in newer Pandas versions makes this behaviour more predictable.
67
Write Pandas code to read a large CSV in chunks and compute total sales per city.
Show answer
# process 1 million rows at a time
import pandas as pd
totals = {}
for chunk in pd.read_csv('sales.csv', usecols=['city', 'amount'], chunksize=1_000_000):
part = chunk.groupby('city')['amount'].sum()
for city, amt in part.items():
totals[city] = totals.get(city, 0) + amt
result = pd.Series(totals).sort_values(ascending=False)
68
Pandas: compute a 7-day rolling average of daily sales for each store.
Show answer
# df has columns: store, date, sales
df['date'] = pd.to_datetime(df['date'])
df = df.sort_values(['store', 'date'])
df['sales_7d_avg'] = (
df.groupby('store')['sales']
.transform(lambda s: s.rolling(7, min_periods=1).mean())
)
69
A 10 million row DataFrame with a city column is using too much memory. How can the category dtype help?
Show answer
The category dtype stores each unique value once and keeps small integer codes for every row. For a column like city with 50 unique values in 10 million rows, this can cut memory by more than 90 percent and speed up groupby and sorting. It helps when a column has low cardinality. It does not help for unique IDs or free text, and adding new categories later needs extra handling.
70
How do you handle dates and time zones in Pandas?
Show answer
Convert strings with pd.to_datetime, giving a format argument for speed and safety. Use the .dt accessor to pull year, month, weekday or hour. For time zones, first tz_localize naive timestamps to their real source zone, then tz_convert to a common zone such as UTC or Asia/Kolkata. Store data in UTC and convert only for reporting. Mixing naive and aware timestamps causes errors, so keep one convention across the pipeline.
71
Your notebook runs on your laptop but fails on a colleague's machine. How do you make the work reproducible?
Show answer
First pin the environment: a requirements.txt or conda environment file with exact versions, or a Docker image. Set random seeds for NumPy, scikit-learn and any framework you use. Remove hard-coded local paths and read them from a config file. Make sure data comes from a shared, versioned location. Finally, restart the kernel and run the notebook top to bottom, so hidden cell order bugs are caught before sharing.
72
A Pandas merge suddenly returns many more rows than the left table had. What happened?
Show answer
The join key is probably not unique on the right side, so each left row matches several right rows and the result multiplies. Check with right['key'].duplicated().sum(). Other causes are null or blank keys matching each other after cleaning, or joining on the wrong column. Fix it by deduplicating or aggregating the right table first, and use merge(..., validate='many_to_one') so Pandas raises an error when the assumption breaks.
73
What is ANOVA and when would you use it?
Show answer
ANOVA, analysis of variance, tests whether three or more group means are equal. It compares variation between groups to variation within groups using an F statistic. For example, comparing average order value across four city tiers. A significant result says at least one mean differs but not which one, so you follow up with post-hoc tests like Tukey's HSD. It assumes independent samples, roughly normal residuals and similar variances.
74
Your A/B test shows a significant lift after 2 days. The product team wants to ship now. What do you say?
Show answer
I would ask them to wait for the planned duration. Checking early and stopping on the first significant result, called peeking, inflates false positives. Two days also misses weekly patterns and novelty effects, where users click a new feature just because it is new. If speed matters, we can use a sequential testing method designed for early stopping. Otherwise we run the full sample size calculated before launch.
75
What is bootstrapping in statistics?
Show answer
Bootstrapping estimates the uncertainty of a statistic by resampling your data with replacement many times, computing the statistic on each resample, and looking at the spread of results. It gives confidence intervals for things with no simple formula, like the median, a ratio or a model metric such as AUC. It needs no normality assumption, but it relies on the sample being representative and can be slow on very large data.
76
Write Python code to compute a 95% bootstrap confidence interval for the mean.
Show answer
import numpy as np
def bootstrap_ci(x, n_boot=10000, seed=42):
rng = np.random.default_rng(seed)
x = np.asarray(x)
means = [rng.choice(x, size=len(x), replace=True).mean() for _ in range(n_boot)]
return np.percentile(means, [2.5, 97.5])
print(bootstrap_ci([12, 15, 9, 20, 14, 11, 18]))
77
What is effect size and why should you report it along with a p-value?
Show answer
Effect size measures how large a difference is, for example the absolute lift in conversion or Cohen's d for means. A p-value only says whether the difference is likely due to chance. With millions of users, a 0.01 percent lift can be significant yet useless for the business. Reporting effect size with a confidence interval tells decision makers both whether the effect is real and whether it is worth acting on.
78
A key metric dropped 8 percent week over week. How do you check whether it is real?
Show answer
First rule out data issues: tracking changes, pipeline delays, missing partitions or a changed metric definition. Then compare with the same week last year and look for holidays or seasonality. Check normal weekly variation to see if 8 percent is unusual. Break the drop down by platform, region, app version and user type to find where it is concentrated. Finally, check releases, campaigns or outages that happened that week.
79
Tell me about a time your analysis contradicted what leadership expected.
Show answer
Situation: leadership believed a discount campaign drove a sales spike. Task: I was asked to measure its impact. Action: I compared similar regions with and without the campaign and found the spike matched a festival week in both. I shared the method, the charts and the limits openly. Result: they cut the next discount budget by half and moved it to retention. I learned to present surprising results with clear evidence and calm framing.
80
What is the difference between hard voting and soft voting in ensembles?
Show answer
In hard voting each model predicts a class and the majority wins. In soft voting each model gives class probabilities, they are averaged, and the class with the highest average wins. Soft voting usually works better because it uses each model's confidence, but it needs calibrated probabilities from all models. Hard voting is simpler and works even with models that only output labels.
81
What is DBSCAN and when is it better than K-means?
Show answer
DBSCAN is a density-based clustering algorithm. It groups points that have at least min_samples neighbours within distance eps, and marks isolated points as noise. Unlike K-means, it does not need the number of clusters, finds clusters of any shape, and handles outliers. It struggles when clusters have very different densities and in high dimensions. It suits location data, like finding delivery hotspots from GPS points.
82
Tell me about a time your model did not perform as expected after deployment.
Show answer
Situation: my demand forecasting model had 8 percent error offline, but 20 percent in the first month live. Task: find out why quickly. Action: I compared live inputs with training data and found a promotions feature arrived a day late in production, so the model saw zeros. I added a data freshness check and a fallback. Result: error returned to 9 percent. Since then I always test with production-like pipelines before launch.
83
Your manager asks whether collecting more training data will improve the model. How do you answer using a learning curve?
Show answer
A learning curve plots training and validation scores against the size of the training set. If both scores are low and close, the model has high bias and needs more features or a more flexible model. If training is high but validation is much lower and the gap narrows as data grows, the model has high variance and more data will help. It tells you whether collecting more data is worth the cost.
84
Your sales forecast scores well in k-fold cross-validation but fails in production. What could be wrong with the validation?
Show answer
Random k-fold mixes future and past rows, so the model trains on data from after the period it predicts. That leaks information and inflates scores. Use time-based splits instead: train on an expanding or rolling window and validate on the next period, like TimeSeriesSplit in scikit-learn. Leave a gap between train and validation when features use lagged values, so the test reflects real forecasting conditions.
85
Write scikit-learn code to get precision, recall and a confusion matrix at a custom threshold of 0.3.
Show answer
from sklearn.metrics import confusion_matrix, precision_score, recall_score
proba = model.predict_proba(X_test)[:, 1]
pred = (proba >= 0.3).astype(int)
print(confusion_matrix(y_test, pred))
print('precision', precision_score(y_test, pred))
print('recall', recall_score(y_test, pred))
86
The business asks you to pick a decision threshold for a churn model. How do you choose it?
Show answer
The default 0.5 is rarely right. I would get the cost of each outcome: the value of saving a churner, the cost of a retention offer, and the team's capacity to contact customers. Then I compute expected profit or precision and recall at many thresholds on validation data and pick the one that maximises value within capacity. I would also check that the chosen threshold stays stable over recent months.
87
How do you evaluate clustering when you have no labels?
Show answer
Use internal metrics: the silhouette score compares how close a point is to its own cluster versus the nearest other cluster, and the Davies-Bouldin index measures cluster overlap. The elbow method looks at within-cluster variance. But the real test is usefulness: are clusters stable across random seeds and samples, can the business describe them, and do they behave differently on outcomes like spend or churn?
88
Your random forest is accurate but too slow at prediction time for a real-time API. What do you do?
Show answer
First measure where the time goes. Then reduce trees and maximum depth and check the accuracy loss. Switch to a gradient boosting model, which often needs fewer, smaller trees. Compile the model with tools like ONNX Runtime or Treelite. Cache features or precompute scores for known users. If accuracy allows, a simpler logistic regression may meet the latency budget easily. Always compare options on the same validation set.
89
Tell me about a time you chose a simpler model over a more complex one.
Show answer
Situation: for a credit risk use case, my deep learning model beat logistic regression by only 0.5 AUC points. Task: recommend one for production. Action: I compared explainability, latency, monitoring effort and regulatory needs, and showed the small gain was within noise across folds. Result: we shipped a scorecard-style logistic regression that the risk team could audit. It launched three weeks earlier and was easy to maintain.
90
Why is cross-entropy preferred over MSE for classification in neural networks?
Show answer
Cross-entropy measures how far predicted probabilities are from the true class and strongly penalises confident wrong predictions. With a softmax or sigmoid output, it gives large, useful gradients when the model is wrong. MSE with sigmoid outputs gives very small gradients when predictions saturate, so learning slows down. Use binary cross-entropy for two classes, categorical cross-entropy for many classes, and MSE for regression targets.
91
What is the difference between SGD and Adam optimisers?
Show answer
SGD updates weights using the gradient times a fixed learning rate, often with momentum. Adam keeps running averages of gradients and their squares, giving each parameter its own adaptive step size. Adam converges fast with little tuning and is the default for transformers. Well-tuned SGD with momentum sometimes generalises better on vision tasks. AdamW, which decouples weight decay, is the common choice for modern deep learning.
92
Tell me about a time you had to learn a new deep learning framework or tool quickly.
Show answer
Situation: my internship project used PyTorch, but I had only used Keras. Task: build a text classifier within two weeks. Action: I followed the official tutorials for two days, rebuilt a small Keras model in PyTorch to compare, and asked a senior to review my training loop. Result: I delivered the model on time with 88 percent F1. I learned that rebuilding something familiar is the fastest way into a new tool.
93
Write a basic PyTorch training loop for one epoch.
Show answer
model.train()
for X, y in train_loader:
X, y = X.to(device), y.to(device)
optimizer.zero_grad()
logits = model(X)
loss = loss_fn(logits, y)
loss.backward()
optimizer.step()
# evaluation
model.eval()
with torch.no_grad():
val_loss = sum(loss_fn(model(Xv.to(device)), yv.to(device)).item() for Xv, yv in val_loader)
94
Your neural network's training loss suddenly becomes NaN. What do you check?
Show answer
Common causes are a learning rate that is too high, exploding gradients, bad input values, and unstable maths like log of zero. I would check the input data for NaN or infinite values, lower the learning rate, add gradient clipping, and make sure the loss uses the stable built-in version with logits. With mixed precision, I would check the loss scaler. Logging gradient norms per step helps find the exact point it breaks.
95
Your image classifier trained on 2,000 images memorises the training set within 5 epochs. What do you try?
Show answer
With so little data, I would use transfer learning from a pretrained model like ResNet or EfficientNet and fine-tune only the top layers first. I would add data augmentation such as flips, crops and colour changes, use dropout and weight decay, and apply early stopping on validation loss. Collecting more labelled images or using a smaller model also helps. I would check for duplicate images across train and validation sets too.
96
What is subword tokenisation like BPE or WordPiece?
Show answer
Subword tokenisers split text into frequent pieces between characters and full words. Byte Pair Encoding starts from characters and repeatedly merges the most frequent pairs. WordPiece, used by BERT, is similar but chooses merges by likelihood. Rare words like unhappiness become un, happi, ness, so there are no unknown words and the vocabulary stays around 30,000 to 100,000. Token counts also decide LLM cost and context use.
97
Training a transformer on a GPU fails with an out-of-memory error. What can you do?
Show answer
Reduce the batch size and use gradient accumulation to keep the same effective batch. Shorten the maximum sequence length. Turn on mixed precision training with float16 or bfloat16. Use gradient checkpointing, which saves memory by recomputing activations. Freeze some layers or switch to a smaller model, or use parameter-efficient fine-tuning like LoRA. Also make sure no old tensors stay on the GPU between runs.
98
A deep network you built from scratch does not learn at all; the loss stays flat from the first epoch. Could weight initialisation be the reason?
Show answer
If weights start too large, activations and gradients explode; if too small, they vanish, and training stalls. If all weights are equal, every neuron learns the same thing. Good schemes keep the variance of activations stable across layers: Xavier or Glorot initialisation for tanh and sigmoid, and He initialisation for ReLU. Frameworks use sensible defaults, but it matters when building custom layers or very deep networks.
99
How do you mentor junior data scientists on your team?
Show answer
Situation: two freshers joined my team and struggled with messy real data. Task: help them deliver independently. Action: I paired with them on the first project, set up weekly code reviews, shared a checklist for validation and leakage, and gave them small end-to-end tasks with clear success metrics. Result: within three months both owned models in production. I focus on teaching how to debug and ask questions, not just giving answers.
100
You target-encoded a city column and validation AUC jumped from 0.72 to 0.95. What do you suspect?
Show answer
Target encoding replaces each category with the average target value for that category, for example the churn rate per city. It handles high-cardinality columns compactly. The main risk is leakage: if a row's own target is included in its encoding, the model sees the answer and overfits. Prevent it with out-of-fold encoding, smoothing toward the global mean for rare categories, and fitting only on training data.
101
Write Pandas code to create lag and rolling features for each customer.
Show answer
df = df.sort_values(['customer_id', 'month'])
g = df.groupby('customer_id')['spend']
df['spend_lag1'] = g.shift(1)
df['spend_lag3'] = g.shift(3)
# shift first so the current month is not included
df['spend_roll3_mean'] = g.transform(lambda s: s.shift(1).rolling(3).mean())
102
What are interaction features and when do they help?
Show answer
Interaction features combine two or more variables, such as price times quantity, or income divided by number of dependents. They help linear models capture effects where one variable changes the impact of another. Tree models can learn some interactions themselves, but explicit ratios often still help. Too many automatic interactions explode the feature count and overfit, so build them from domain logic and validate each one.
103
Your model's most important feature is customer_id. What is wrong?
Show answer
An ID has no real meaning, so high importance means the model is memorising individual customers. It often happens when the same customers appear in both train and test, or when IDs were assigned in time order and act as a proxy for date or signup cohort. I would drop the ID, use a group-based split by customer, and if timing matters, add an explicit feature like account age instead.
104
How do you turn text columns into features for a tabular model?
Show answer
Simple options are length, word count and keyword flags. TF-IDF vectors capture important words and work well with linear models. For meaning, use sentence embeddings from a pretrained model and optionally reduce them with PCA. You can also train a separate text classifier and feed its predicted score as one feature. Fit any vocabulary or encoder on training data only to avoid leakage.
105
Write a scikit-learn ColumnTransformer that preprocesses numeric and categorical columns.
Show answer
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import make_pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
pre = ColumnTransformer([
('num', make_pipeline(SimpleImputer(strategy='median'), StandardScaler()), num_cols),
('cat', make_pipeline(SimpleImputer(strategy='most_frequent'),
OneHotEncoder(handle_unknown='ignore')), cat_cols)
])
model = make_pipeline(pre, LogisticRegression(max_iter=1000))
106
What is recursive feature elimination?
Show answer
Recursive feature elimination trains a model, ranks features by importance or coefficient size, removes the weakest ones, and repeats until the desired number remains. RFECV picks the number of features using cross-validation. It considers features together rather than one at a time. The downsides are high compute cost on large data and unstable results when features are correlated, so check stability across folds.
107
How would you build RFM features for a customer churn model?
Show answer
RFM stands for recency, frequency and monetary value. From the transactions table, for each customer as of a cut-off date: recency is days since the last purchase, frequency is the number of orders in the last 90 days, and monetary is total or average spend in that window. Compute them using only data before the cut-off, add trends like spend this month versus last, and the label from the period after.
108
How do you work with domain experts to find good features?
Show answer
Situation: on an insurance claims model, my first features gave average results. Task: improve fraud detection. Action: I sat with two claim investigators for a day and asked what made them suspicious. They mentioned claims filed soon after policy start and repeated garage names. I turned these into features and checked them with the team. Result: recall at the same precision rose by 12 points. Experts often know signals the data hides.
109
Write Python code to compute the haversine distance between customer and store locations.
Show answer
import numpy as np
def haversine_km(lat1, lon1, lat2, lon2):
lat1, lon1, lat2, lon2 = map(np.radians, [lat1, lon1, lat2, lon2])
dlat, dlon = lat2 - lat1, lon2 - lon1
a = np.sin(dlat / 2) ** 2 + np.cos(lat1) * np.cos(lat2) * np.sin(dlon / 2) ** 2
return 6371 * 2 * np.arcsin(np.sqrt(a))
df['dist_km'] = haversine_km(df.cust_lat, df.cust_lon, df.store_lat, df.store_lon)
110
After one-hot encoding you have 2,000 features and training is slow. What do you do?
Show answer
Find which columns caused the explosion, usually a few high-cardinality categoricals. Group rare categories into Other, or switch those columns to frequency, target or hashing encoding. Use sparse matrices so zeros take no memory. Tree libraries like LightGBM and CatBoost can handle categoricals natively without one-hot. Then remove near-constant columns and check that validation scores hold up with the smaller feature set.
111
Tell me about a feature you engineered that significantly improved a model.
Show answer
Situation: our delivery time model kept underestimating late-night orders. Task: reduce large errors. Action: I noticed restaurant prep time depended on how busy the kitchen was, so I built a feature for active orders at that restaurant in the last 30 minutes, computed point in time. Result: MAE dropped by 14 percent and late-night errors fell the most. It taught me to study the worst errors before adding generic features.
112
What are polynomial features and what is the risk of using them?
Show answer
Polynomial features add powers and products of existing features, like x squared and x times y, so a linear model can fit curves and interactions. The risk is that the number of features grows very fast with degree and input count, causing overfitting, slow training and multicollinearity. Use a low degree, scale features first, and add regularisation such as Ridge or Lasso to keep coefficients under control.
113
Write SQL to find the top 3 products by revenue in each category.
Show answer
WITH rev AS (
SELECT category, product_id, SUM(price * qty) AS revenue
FROM order_items
GROUP BY category, product_id
), ranked AS (
SELECT *, ROW_NUMBER() OVER (PARTITION BY category ORDER BY revenue DESC) AS rn
FROM rev
)
SELECT category, product_id, revenue
FROM ranked
WHERE rn <= 3;
114
Write SQL to compute a 7-day rolling average of daily active users.
Show answer
WITH daily AS (
SELECT activity_date, COUNT(DISTINCT user_id) AS dau
FROM events
GROUP BY activity_date
)
SELECT activity_date, dau,
AVG(dau) OVER (ORDER BY activity_date
ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS dau_7d_avg
FROM daily
ORDER BY activity_date;
115
Write SQL to calculate month-over-month revenue growth.
Show answer
WITH monthly AS (
SELECT DATE_TRUNC('month', order_date) AS month, SUM(amount) AS revenue
FROM orders
GROUP BY 1
)
SELECT month, revenue,
LAG(revenue) OVER (ORDER BY month) AS prev_revenue,
ROUND(100.0 * (revenue - LAG(revenue) OVER (ORDER BY month))
/ NULLIF(LAG(revenue) OVER (ORDER BY month), 0), 2) AS mom_pct
FROM monthly;
116
What is a CTE and what is a recursive CTE?
Show answer
A CTE, common table expression, is a named temporary result defined with WITH and used in the query that follows. It breaks complex SQL into readable steps and can be referenced more than once. A recursive CTE refers to itself: an anchor query starts the result and a recursive part keeps adding rows until none are added. It is used for hierarchies like employee-manager chains or generating date series.
117
You added an index but the query is still slow. How do indexes work, and when do they fail to help or even hurt?
Show answer
An index is a separate sorted structure, usually a B-tree, that lets the database find rows without scanning the whole table, like a book index. It speeds up filters, joins and sorting on the indexed columns. Indexes cost storage and slow down inserts, updates and deletes because each index must be updated. They may be ignored for low-selectivity columns or when a function is applied to the column in WHERE.
118
A query using NOT IN returns zero rows even though matches should exist. What is the likely cause, and how do NULLs behave in SQL?
Show answer
NULL means unknown, so any comparison with NULL, even NULL = NULL, gives unknown and the row is filtered out. Use IS NULL or IS NOT NULL instead. Aggregates like SUM, AVG and COUNT(column) skip NULLs, so AVG may be computed over fewer rows than you expect. NOT IN with a subquery that contains a NULL returns no rows. COALESCE replaces NULLs with a default value.
119
After joining orders with a promotions table, total revenue doubled. What went wrong?
Show answer
The join created duplicate order rows, because some orders match more than one promotion row, so each amount is counted several times. Check by comparing COUNT(*) with COUNT(DISTINCT order_id) after the join. Fix it by aggregating the promotions table to one row per order first, picking one promotion with ROW_NUMBER, or summing revenue before the join. Always check row counts after every join.
120
Write SQL to find customers who ordered in January 2026 but not in February 2026.
Show answer
SELECT DISTINCT o.customer_id
FROM orders o
WHERE o.order_date >= '2026-01-01' AND o.order_date < '2026-02-01'
AND NOT EXISTS (
SELECT 1 FROM orders f
WHERE f.customer_id = o.customer_id
AND f.order_date >= '2026-02-01' AND f.order_date < '2026-03-01'
);
121
What is normalisation, and why are analytics tables often denormalised?
Show answer
Normalisation splits data into related tables to remove redundancy, so each fact is stored once, which suits transactional systems with many updates. Analytics workloads mostly read large volumes, and joins across many tables are slow and error-prone. So warehouses often use denormalised, wide tables or star schemas that repeat some attributes. The trade-off is more storage and the need to keep copies consistent through pipelines.
122
What is the difference between a star schema and a snowflake schema?
Show answer
A star schema has a central fact table, like sales with amounts and foreign keys, surrounded by denormalised dimension tables such as product, customer and date. A snowflake schema normalises dimensions further, for example product links to a separate category table. Star schemas need fewer joins and are faster and simpler for analysts. Snowflake schemas save storage and reduce duplication but make queries more complex.
123
Tell me about a time you had to deliver data under an urgent deadline.
Show answer
Situation: the leadership team needed festive-sale numbers by city for a 9 am review, and the dashboard pipeline had failed overnight. Task: deliver accurate numbers in three hours. Action: I queried the raw orders tables directly, reused validated logic from the dashboard SQL, cross-checked totals with finance's daily figure and flagged two cities with partial data. Result: the review went ahead on time, and I later added pipeline failure alerts.
124
How do you pull large data from a SQL database into Pandas efficiently?
Show answer
Push as much work as possible into SQL: filter, aggregate and select only needed columns before pulling. Use pd.read_sql with a SQLAlchemy engine and chunksize to stream large results. For very large extracts, export to Parquet from the warehouse and read that, which is much faster than row-by-row transfer. Set dtypes like category early, and never use SELECT star on a big table from a notebook.
125
Write SQL to compute a running total of spend for each user.
Show answer
SELECT user_id, order_date, amount,
SUM(amount) OVER (
PARTITION BY user_id
ORDER BY order_date
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
) AS running_spend
FROM orders
ORDER BY user_id, order_date;
126
Your SQL result does not match the number on the official dashboard. How do you resolve it?
Show answer
I would first align definitions: date range, time zone, filters like test users or cancelled orders, and whether the metric counts users or events. Then compare at a smaller grain, by day or by city, to find where the gap starts. I would read the dashboard's underlying query and check data refresh times. Once found, I would document the agreed definition so the same mismatch does not repeat.
127
What is the difference between a subquery and a correlated subquery?
Show answer
A normal subquery runs once on its own and its result is used by the outer query, for example WHERE price > (SELECT AVG(price) FROM products). A correlated subquery refers to columns of the outer query, so it is logically evaluated once per outer row, for example comparing each product with its own category average. Correlated subqueries can be slow; joins or window functions are usually faster alternatives.
128
Describe a time you had to push back on or prioritise many ad-hoc data requests.
Show answer
Situation: as the only analyst for three teams, I received 15 ad-hoc SQL requests a week and my model work stalled. Task: protect focus without upsetting stakeholders. Action: I grouped requests, built a self-serve dashboard covering the 5 most common questions, and set up a simple weekly priority review with the managers. Result: ad-hoc requests dropped by 60 percent and the model shipped a month later as planned.
129
What are embeddings and vector databases, and why do LLM apps use them?
Show answer
An embedding model turns text into a vector so that similar meanings are close in space. A vector database stores these vectors with metadata and finds the nearest ones to a query vector quickly using approximate nearest neighbour indexes like HNSW. LLM apps use them for semantic search and retrieval-augmented generation, finding the most relevant documents to put into the prompt. Examples include pgvector, FAISS and dedicated vector stores.
130
What is chunking in RAG and how do you choose a chunk size?
Show answer
Chunking splits documents into smaller pieces before embedding them, so retrieval returns focused passages that fit in the prompt. Small chunks, like 200 tokens, are precise but lose context; large chunks, like 1,000 tokens, keep context but add noise and cost. Split on natural boundaries such as headings and paragraphs, add some overlap, and keep metadata like title and section. Then tune chunk size using retrieval metrics on real questions.
131
Your LLM assistant confidently quotes a policy clause that does not exist. Why does this happen and how do you reduce it?
Show answer
Hallucination is when a model produces confident text that is false or not supported by any source. It happens because the model generates plausible tokens rather than checking facts. Reduce it by grounding answers in retrieved documents, asking the model to cite sources and say I do not know when context is missing, using low temperature, and validating outputs with rules or a second checking step. Measure it with a labelled evaluation set.
132
Write Python code to find the top 3 most similar documents to a query using cosine similarity on embeddings.
Show answer
import numpy as np
def top_k(query_vec, doc_vecs, k=3):
q = query_vec / np.linalg.norm(query_vec)
d = doc_vecs / np.linalg.norm(doc_vecs, axis=1, keepdims=True)
scores = d @ q
idx = np.argsort(-scores)[:k]
return list(zip(idx, scores[idx]))
# doc_vecs shape: (n_docs, dim), query_vec shape: (dim,)
133
What is function calling or tool use in LLMs?
Show answer
Function calling lets an LLM respond with a structured request to call a tool you defined, such as get_order_status with an order_id, instead of plain text. Your code runs the function, returns the result to the model, and the model uses it in its final answer. This connects LLMs to live data, calculators and APIs. You must validate arguments, limit which tools exist and handle errors, because the model can choose wrong inputs.
134
What is an AI agent in the context of LLMs?
Show answer
An AI agent is an LLM that works in a loop: it plans a step, calls a tool, reads the result and decides the next step until the goal is met. For example, it might query a database, run Python on the result and write a summary. Agents handle multi-step tasks but are less predictable, slower and costlier than a single call. Good designs limit tools and steps, log every action and keep a human check for risky actions.
135
How do you get reliable JSON output from an LLM?
Show answer
Use the provider's structured output or JSON schema mode when available, which constrains the output to your schema. Otherwise, describe the exact schema in the prompt with an example, and use low temperature. Always parse and validate the response in code, for example with Pydantic, and on failure retry with the error message included. Keep schemas small and flat, and use enums for fields with fixed values.
136
Write a prompt template for classifying customer support tickets.
Show answer
System: You classify support tickets for an e-commerce company. Categories: payment, delivery, refund, account, other. Rules: choose exactly one category; if unsure, choose other; do not invent details. Return JSON with fields category and reason, reason under 20 words. Then two short examples showing a ticket and the expected JSON. User: Ticket: {ticket_text}. Keep the template fixed and test it on 100 labelled tickets before rollout.
137
What is quantisation of LLMs?
Show answer
Quantisation stores model weights in lower precision, such as 8-bit or 4-bit integers instead of 16-bit floats. A 7-billion parameter model drops from about 14 GB to around 4 GB at 4-bit, so it fits on smaller GPUs or even laptops and runs faster. The cost is some loss in quality, usually small at 8-bit and noticeable on hard tasks at very low bits. Common formats include GPTQ, AWQ and GGUF.
138
Your LLM app takes 8 seconds to respond and users complain. What do you do?
Show answer
Break down where the time goes: retrieval, model time to first token and generation length. Stream tokens so users see output immediately. Cut prompt and output length, run retrieval and other calls in parallel, and cache frequent answers. Use a smaller or faster model for simple requests, and avoid chains of several LLM calls where one would do. Set a latency budget per step and monitor p95, not just averages.
139
Users say your internal chatbot gives outdated policy answers. What is the likely cause and fix?
Show answer
The knowledge index probably still holds old document versions, or the model is answering from its training data instead of retrieved context. Fix the ingestion pipeline to re-index on document change and remove superseded versions. Store effective dates in metadata and filter or boost the latest. Instruct the model to answer only from provided context and show the source with its date, so users can verify it.
140
What are guardrails in LLM applications?
Show answer
Guardrails are checks around the model that keep inputs and outputs safe and on-topic. Input guardrails detect prompt injection, abuse or off-topic requests. Output guardrails check for personal data, toxic content, policy violations or invalid formats before the answer reaches the user. They can be rules, classifiers or a second LLM call. Good guardrails also define fallback responses and log blocked cases for review.
141
Tell me about a GenAI project you built and what you learned from it.
Show answer
Situation: our HR team answered the same leave and policy questions daily. Task: build an internal assistant in six weeks. Action: I built a RAG pipeline over policy PDFs, added citations, created a 150-question evaluation set with HR, and iterated on chunking and reranking. Result: answer accuracy reached 91 percent and HR queries dropped by 40 percent. I learned that the evaluation set mattered more than the choice of model.
142
Tell me about a time you explained the limits of GenAI to business stakeholders.
Show answer
Situation: sales leaders wanted an LLM to auto-send pricing quotes to clients. Task: guide them to a safe plan. Action: I demoed real failure cases where the model invented discounts, showed accuracy numbers from a test set and proposed a draft-and-approve flow with fixed price data from our system. Result: they agreed, quote drafting time fell by half, and no wrong prices reached clients.
143
What is experiment tracking and why use a tool like MLflow?
Show answer
Experiment tracking records each training run's parameters, code version, data version, metrics and output artefacts. Tools like MLflow or Weights and Biases store this in one place, so you can compare runs, reproduce the best model and see why something improved. Without it, results live in notebook outputs and memory. MLflow also offers a model registry to move approved models through staging and production.
144
Write a FastAPI endpoint that serves predictions from a saved scikit-learn model.
Show answer
from fastapi import FastAPI
from pydantic import BaseModel
import joblib
import pandas as pd
app = FastAPI()
model = joblib.load('model.joblib')
class Customer(BaseModel):
tenure: int
monthly_spend: float
plan: str
@app.post('/predict')
def predict(c: Customer):
X = pd.DataFrame([c.model_dump()])
return {'churn_probability': float(model.predict_proba(X)[0, 1])}
145
Write a Dockerfile for a Python model API.
Show answer
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app/ ./app
COPY model.joblib .
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
146
What does CI/CD mean for machine learning projects?
Show answer
Continuous integration runs automatic checks on every code change: unit tests for feature code, data validation, linting and a quick training run on sample data. Continuous delivery packages the model and service, for example as a Docker image, and deploys it through staging to production after evaluation gates, such as the new model beating the current one on a holdout set. Some teams add continuous training, which retrains on new data automatically.
147
What is the difference between saving a model with pickle, joblib and ONNX?
Show answer
pickle serialises Python objects; it is simple but tied to Python and library versions, and loading untrusted pickle files is a security risk. joblib is similar but more efficient for objects with large NumPy arrays, like scikit-learn models. ONNX is an open, language-neutral format that runs in optimised runtimes across languages and hardware, useful for fast serving. Always store library versions alongside pickle or joblib files.
148
Your model performance degrades every few months. Would you retrain on a fixed schedule or based on triggers?
Show answer
Scheduled retraining, like monthly, is simple and predictable but may be too late after a sudden change or wasteful when nothing changed. Trigger-based retraining runs when monitoring shows drift or a performance drop, which reacts faster but needs reliable monitoring. Many teams combine both: a regular schedule plus triggers for big shifts. Either way, a retrained model must pass the same evaluation gate before replacing the current one.
149
How do you make a machine learning experiment reproducible?
Show answer
Version the code with Git, the data with a tool like DVC or dated snapshots, and the environment with pinned requirements or Docker. Set random seeds for every library. Log parameters, metrics and the exact data version for each run with a tracker like MLflow. Keep preprocessing inside a pipeline rather than manual notebook steps. Then anyone should be able to rebuild the same model from a commit and a data version.
150
What should you log for a production prediction service?
Show answer
For each request, log a request ID, timestamp, model version, input features or a hash of them, the prediction and score, and latency. Log errors and fallback use separately. Do not log raw personal data unless needed and allowed; mask it. Aggregate these logs into dashboards for traffic, latency percentiles, error rate and score distribution. Joining logged predictions with later outcomes gives the data for monitoring and retraining.
151
Your deployed model gives different predictions than the same model in your notebook. Why?
Show answer
Usually preprocessing differs between training and serving. Check that the same pipeline object was saved, not just the estimator, and that category order, missing value handling and scaling match. Compare library versions, feature types like strings versus integers, column order, and time zones in date features. Send the same few rows through both paths and compare features step by step to find the first mismatch.
152
Write pytest unit tests for a feature function that computes days since last purchase.
Show answer
import pandas as pd
from features import days_since_last_purchase
def test_basic_case():
df = pd.DataFrame({'customer_id': [1, 1], 'order_date': pd.to_datetime(['2026-01-01', '2026-01-10'])})
out = days_since_last_purchase(df, as_of='2026-01-15')
assert out.loc[1] == 5
def test_no_future_orders_used():
df = pd.DataFrame({'customer_id': [1], 'order_date': pd.to_datetime(['2026-02-01'])})
out = days_since_last_purchase(df, as_of='2026-01-15')
assert out.empty
153
Tell me about a time you deployed a model under a tight deadline.
Show answer
Situation: a pricing model had to go live before a sale in ten days. Task: deploy safely in that window. Action: I cut scope to batch scoring instead of real-time, reused our standard Docker and Airflow template, wrote tests for the key features and set up a dashboard for score distribution. Result: it went live two days early and ran without incidents during the sale. Real-time scoring followed a month later.
154
Tell me about a time you disagreed with a stakeholder about which metric to optimise.
Show answer
Situation: marketing wanted the lead model judged on overall accuracy. Task: agree on a metric that matched the goal. Action: I showed that accuracy rewarded predicting no conversion for everyone, then proposed precision in the top 1,000 leads per week, which matched the sales team's call capacity, with a simple chart. Result: they agreed, and the model with the better top-1,000 precision added 18 percent more conversions.
155
What do ACF and PACF plots tell you about a time series?
Show answer
The ACF plot shows correlation between the series and its own lags, including indirect effects through intermediate lags. The PACF shows the correlation at each lag after removing the effect of shorter lags. For a stationary series, a PACF that cuts off after lag p suggests an AR(p) term, and an ACF that cuts off after lag q suggests an MA(q) term. Spikes at lag 7 or 12 reveal weekly or yearly seasonality.
156
What is the Augmented Dickey-Fuller test?
Show answer
The ADF test checks whether a series has a unit root, which means it is non-stationary. The null hypothesis is that a unit root exists. A small p-value, say below 0.05, lets you reject the null and treat the series as stationary. It is often paired with the KPSS test, whose null is the opposite, stationarity. Using both gives a more reliable conclusion before choosing how much to difference.
157
Explain exponential smoothing and the Holt-Winters method.
Show answer
Simple exponential smoothing forecasts with a weighted average of past values where weights decay exponentially, so recent points matter more. Holt's method adds a smoothed trend component. Holt-Winters adds a seasonal component as well, either additive or multiplicative. Each component has a smoothing parameter between 0 and 1 that is fitted to the data. These models are fast, easy to explain and strong baselines for retail and demand data.
158
What is SARIMA and how does it extend a plain ARIMA model?
Show answer
SARIMA adds seasonal terms to ARIMA. It is written ARIMA(p,d,q)(P,D,Q)m, where the second set are seasonal AR, differencing and MA orders, and m is the season length, such as 7 for daily data with a weekly cycle or 12 for monthly data. SARIMAX further allows external regressors like price or holidays. It handles one seasonal period well but struggles with multiple seasonalities, such as daily plus yearly.
159
What is MAPE and when is it a poor metric for forecast accuracy?
Show answer
MAPE is the mean of absolute errors divided by actual values, expressed as a percentage. It is easy to explain to business teams, but it breaks when actuals are zero or near zero, because the percentage explodes. It also penalises over-forecasts more than under-forecasts and weights small items heavily. For intermittent or low-volume series prefer MAE, weighted MAPE or MASE, which compares error to a naive baseline.
160
Your daily sales forecast badly misses the Diwali spike every year. How do you fix it?
Show answer
The model is treating Diwali as noise because the festival moves on the calendar and a fixed yearly seasonality does not align. Add explicit holiday features: a flag for Diwali and a few days before and after, since buying starts early. Include other regional festivals and sale events too. Models like Prophet or gradient boosting with holiday features handle this well. Then backtest specifically on past festive periods to confirm the fix.
161
You must forecast demand for a new product that has only three weeks of history. What do you do?
Show answer
Three weeks is too little to learn seasonality, so borrow information. Find similar past launches by category, price band and channel, and use their early sales curves as analogues. A global model trained across many products, with product attributes as features, can also forecast a cold-start item. Combine this with planned marketing spend and business judgement, show a wide interval, and update the forecast weekly as real data arrives.
162
Most of your products sell zero units on most days, with occasional sales. Which forecasting approach suits this?
Show answer
This is intermittent demand, and standard models like ARIMA forecast a small constant that is useless for stocking. Croston's method and its SBA variant forecast the size of non-zero demand and the interval between sales separately. Alternatives are aggregating to weekly level, or using a global gradient boosting model with Tweedie or Poisson loss. Evaluate with MAE or MASE, not MAPE, because zero actuals break percentage errors.
163
Your forecast uses actual daily temperature as a feature and looks great in backtesting, but it performs worse in production. Why?
Show answer
In backtesting you used the actual temperature for future days, which you will never know at forecast time. In production the model gets a weather forecast, which has its own error, so performance drops. This is a subtle form of leakage. Fix the backtest by using the weather forecast that was available on each forecast date, or lagged actuals, so offline results match what production can really achieve.
164
Write Python code to run an ADF stationarity test and difference the series if needed.
Show answer
from statsmodels.tsa.stattools import adfuller
def is_stationary(s, alpha=0.05):
p_value = adfuller(s.dropna())[1]
return p_value < alpha
series = s.copy()
d = 0
while not is_stationary(series) and d < 2:
series = series.diff()
d += 1
print('differencing order d =', d)
165
Write Python code to evaluate a forecasting model with expanding-window time series cross-validation.
Show answer
import numpy as np
from sklearn.model_selection import TimeSeriesSplit
from sklearn.metrics import mean_absolute_error
tscv = TimeSeriesSplit(n_splits=5, test_size=28)
scores = []
for tr, te in tscv.split(X):
model.fit(X.iloc[tr], y.iloc[tr])
pred = model.predict(X.iloc[te])
scores.append(mean_absolute_error(y.iloc[te], pred))
print(np.mean(scores), scores)
166
Write Python code to fit a Holt-Winters model and forecast the next 12 months.
Show answer
from statsmodels.tsa.holtwinters import ExponentialSmoothing
s = df.set_index('month')['sales'].asfreq('MS')
train, test = s[:-12], s[-12:]
model = ExponentialSmoothing(
train, trend='add', seasonal='mul',
seasonal_periods=12
).fit()
forecast = model.forecast(12)
mae = (forecast - test).abs().mean()
print(forecast.round(1), 'MAE:', round(mae, 1))
167
Tell me about a time your forecast was badly wrong. How did you handle it?
Show answer
Situation: my weekly demand forecast under-predicted a category by 30 percent for a month. Task: explain the miss and stop it recurring. Action: I compared forecast and actuals by region and found a new distribution channel had launched that the model knew nothing about. I told the planning team quickly, added channel as a feature and set up a weekly error review. Result: error returned to normal and planners trusted the alerts.
168
Pandas: find customers whose monthly spend increased for three consecutive months.
Show answer
m = (df.assign(month=df['date'].dt.to_period('M'))
.groupby(['customer_id', 'month'])['amount'].sum()
.reset_index()
.sort_values(['customer_id', 'month']))
m['up'] = (m.groupby('customer_id')['amount'].diff() > 0).astype(int)
# three rises in a row = three consecutive increases
m['streak'] = (m.groupby('customer_id')['up']
.rolling(3).sum().reset_index(level=0, drop=True))
result = m.loc[m['streak'] == 3, 'customer_id'].unique()
169
Tell me about a time you found a mistake in your own analysis after sharing it.
Show answer
Situation: after presenting a retention report, I noticed I had counted test accounts as users. Task: correct it before decisions were made. Action: I reran the numbers that evening, informed my manager and the audience with a short note showing old and new figures and what changed, and added a filter and a check to the shared query. Result: the conclusion held with smaller numbers, and the team valued the transparency.
170
What is an isolation forest and how does it detect anomalies?
Show answer
An isolation forest builds many random trees that split data on random features at random thresholds. Anomalies are rare and different, so they get isolated in very few splits, giving a short average path length. Normal points need many splits. The anomaly score comes from that path length. It is fast, works without labels and scales to large data, but you must choose a contamination rate or score threshold carefully.
171
Your credit model has good overall AUC but performs much worse for applicants from one region. What do you do?
Show answer
First confirm with per-segment metrics and enough samples, since small groups are noisy. Then check why: fewer training examples from that region, different feature distributions, missing data or labels that reflect past biased decisions. Fixes include collecting more data, reweighting, segment-aware features or a separate calibration. Also review fairness and regulatory requirements, and agree with risk and compliance teams on acceptable performance gaps before launch.
172
Write scikit-learn code to calibrate a gradient boosting classifier's probabilities and check the result.
Show answer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.calibration import CalibratedClassifierCV
from sklearn.metrics import brier_score_loss
base = HistGradientBoostingClassifier()
cal = CalibratedClassifierCV(base, method='isotonic', cv=5)
cal.fit(X_train, y_train)
prob = cal.predict_proba(X_test)[:, 1]
print('Brier:', brier_score_loss(y_test, prob))
173
Tell me about a time you had to build a model when the requirements were unclear.
Show answer
Situation: a sales head asked for a model to find good leads but could not define good. Task: turn this into a clear target. Action: I met the team, looked at past deals, and proposed converted within 60 days as the label, with expected revenue as a secondary view. I shared a quick baseline in a week to get feedback. Result: the definition was agreed, and the final model raised conversion in the top decile.
174
What is the difference between semantic, instance and panoptic segmentation?
Show answer
Semantic segmentation gives each pixel a class, such as road or car, but does not separate two cars. Instance segmentation separates individual objects of countable classes, giving each car its own mask. Panoptic segmentation combines both: every pixel gets a class, and countable objects also get an instance ID, while background stuff like sky stays semantic. Autonomous driving and robotics often need panoptic or instance output.
175
What is mean Average Precision (mAP) in object detection?
Show answer
For each class, detections are ranked by confidence and a precision-recall curve is computed, where a detection is a true positive if its IoU with an unmatched ground truth box passes a threshold. Average Precision is the area under that curve. mAP is the mean across classes. mAP@0.5 uses IoU 0.5, while mAP@0.5:0.95 averages over thresholds from 0.5 to 0.95 and rewards tighter boxes.
176
How do one-stage and two-stage object detectors differ?
Show answer
Two-stage detectors, like Faster R-CNN, first propose candidate regions and then classify and refine each one, which gives strong accuracy, especially for small objects, but is slower. One-stage detectors, like YOLO and SSD, predict boxes and classes directly over a dense grid in a single pass, making them fast enough for real-time video. Modern one-stage models have closed most of the accuracy gap, so they dominate edge and video use cases.
177
What is the receptive field of a neuron in a CNN?
Show answer
The receptive field is the region of the input image that can influence one activation. Early layers see small patches like edges, and deeper layers see larger regions, enabling recognition of whole objects. It grows with kernel size, depth, stride and pooling, and dilated convolutions increase it without extra parameters. If the receptive field is smaller than the objects you care about, the network cannot capture their full context.
178
Why do residual connections make very deep networks easier to train?
Show answer
A residual block adds its input to its output, so the layers learn a residual F(x) and the block outputs F(x) plus x. If extra layers are not useful, they can learn to output near zero and the block behaves like identity. The skip path also lets gradients flow directly to early layers, reducing vanishing gradients. This is why ResNets with 50 or 100-plus layers train well where plain deep networks degrade.
179
What is U-Net and why is it popular for segmentation?
Show answer
U-Net is an encoder-decoder network. The encoder downsamples to capture context, and the decoder upsamples back to full resolution to produce a per-pixel mask. Skip connections copy encoder feature maps to the matching decoder level, so fine spatial detail lost during downsampling is recovered. It works well with limited labelled data, which made it the standard for medical imaging and industrial defect segmentation, and it is also the backbone of many diffusion models.
180
You are building defect detection for a factory line where only 1 in 1,000 images shows a defect. How do you approach it?
Show answer
With so few defects, a plain classifier will learn to say no defect. Collect every defect image possible and oversample them or use class-weighted or focal loss. Consider anomaly detection trained only on good parts, which flags anything unusual. Use augmentation that mimics real variation in lighting and position. Evaluate with recall at a fixed false alarm rate, since missing a defect usually costs far more than a manual recheck.
181
Your product image classifier works on studio catalogue photos but fails on photos users take with their phones. Why, and what do you do?
Show answer
This is domain shift: user photos have clutter, blur, poor lighting and odd angles that the model never saw. Collect and label a sample of real user photos and evaluate on them separately. Add augmentation for blur, noise, colour jitter, rotation and random crops, and fine-tune on a mix of studio and real images. Monitor accuracy on user photos in production, since that is the distribution that matters.
182
Your classifier scores 98 percent on validation, but Grad-CAM shows it focuses on a watermark rather than the object. What is happening?
Show answer
The model has learned a shortcut: the watermark correlates with the label in your dataset, perhaps because images of one class came from a single source. It will fail when that correlation breaks. Remove or crop the watermarks, rebalance sources across classes, and create a validation set from different sources. Keep using saliency maps and slice-based evaluation to catch other spurious cues such as backgrounds or text.
183
You need to extract fields from scanned invoices that come in hundreds of different layouts. How would you design it?
Show answer
Fixed templates will not scale to hundreds of layouts. Start with preprocessing: deskew, denoise and detect text regions. Run OCR to get words with their positions. Then use a layout-aware document model, or a vision-language model, to map words to fields like invoice number, GSTIN and total. Add validation rules, such as GSTIN format and line items summing to the total, and send low-confidence documents to human review.
184
Write a Python function to compute IoU between two bounding boxes given as (x1, y1, x2, y2).
Show answer
def iou(a, b):
ix1, iy1 = max(a[0], b[0]), max(a[1], b[1])
ix2, iy2 = min(a[2], b[2]), min(a[3], b[3])
inter = max(0, ix2 - ix1) * max(0, iy2 - iy1)
area_a = (a[2] - a[0]) * (a[3] - a[1])
area_b = (b[2] - b[0]) * (b[3] - b[1])
union = area_a + area_b - inter
return inter / union if union > 0 else 0.0
print(iou((0, 0, 10, 10), (5, 5, 15, 15))) # 0.142...
185
Write PyTorch code to fine-tune a pretrained ResNet-18 for a 5-class problem by replacing its final layer.
Show answer
import torch, torch.nn as nn
from torchvision import models
model = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)
for p in model.parameters():
p.requires_grad = False # freeze backbone
model.fc = nn.Linear(model.fc.in_features, 5)
opt = torch.optim.AdamW(model.fc.parameters(), lr=1e-3)
loss_fn = nn.CrossEntropyLoss()
# train head first, then unfreeze later layers with a lower lr
186
Write NumPy code to compute the Dice coefficient between a predicted and a true binary mask.
Show answer
import numpy as np
def dice(pred, true, eps=1e-7):
pred = pred.astype(bool)
true = true.astype(bool)
inter = np.logical_and(pred, true).sum()
return (2 * inter + eps) / (pred.sum() + true.sum() + eps)
# Dice = 2|A and B| / (|A| + |B|); 1 means perfect overlap
187
Tell me about a computer vision project where data quality was the biggest challenge.
Show answer
Situation: a shelf-detection model plateaued at low accuracy. Task: find out why and fix it. Action: I reviewed 500 errors and found that a third were labelling mistakes, with inconsistent boxes for partly hidden products. I rewrote the guidelines with visual examples, had a second annotator review hard images and relabelled the worst batch. Result: mAP rose by 9 points without changing the model, and labelling rework dropped sharply.
188
What is the difference between layer normalisation and batch normalisation?
Show answer
Batch normalisation normalises each feature using the mean and variance across the batch, so it depends on batch size and behaves differently in training and inference. Layer normalisation normalises across the features of each single example, so it works with any batch size and with variable-length sequences. That is why transformers use layer norm, often in the pre-norm position, while CNNs traditionally use batch norm.
189
You must classify support messages into 40 categories, but some categories have only 20 examples. What do you do?
Show answer
Start with a pretrained sentence embedding model plus a simple classifier, or fine-tune a small transformer with class weights. For the rare categories, try few-shot approaches such as SetFit or an LLM with examples in the prompt, and generate or collect more examples. Merge categories that are rarely distinguishable. Report per-class F1 rather than overall accuracy, and route low-confidence predictions to a human queue.
190
Tell me about a time you had to explain a deep learning model's mistake to a client or stakeholder.
Show answer
Situation: our document classifier mislabelled several legal notices, and the client was upset. Task: explain clearly and restore confidence. Action: I showed the misclassified examples, explained that they used a format absent from training data, and avoided jargon. I proposed adding those samples, a confidence threshold for human review and weekly error reports. Result: errors on that format fell sharply and the client extended the project.
191
Write SQL to show each customer's first order date, last order date and the gap between them in days.
Show answer
SELECT
customer_id,
MIN(order_date) AS first_order,
MAX(order_date) AS last_order,
MAX(order_date) - MIN(order_date) AS days_between,
COUNT(*) AS orders
FROM orders
GROUP BY customer_id
ORDER BY days_between DESC;
-- PostgreSQL: DATE minus DATE gives days; use DATEDIFF in other engines
192
What is the randomisation unit in an experiment, and why does it matter?
Show answer
The randomisation unit is what you assign to variants: a user, a session, a device, a page view or even a city. Choosing user-level keeps each person's experience consistent and avoids contamination. The analysis unit should match the randomisation unit, because if you randomise by user but analyse by page view, observations are correlated and standard errors come out too small, producing false positives. Use the delta method or bootstrap when they differ.
193
What is the minimum detectable effect (MDE) in experiment design?
Show answer
The MDE is the smallest true effect your test can reliably detect given the sample size, baseline rate, significance level and power. If baseline conversion is 5 percent and your MDE is 10 percent relative, you can detect a move to 5.5 percent but will likely miss smaller lifts. Choose the MDE from business value: the smallest change worth shipping. A smaller MDE needs a much larger sample, roughly growing with one over MDE squared.
194
What are the novelty effect and the primacy effect in A/B tests?
Show answer
The novelty effect is when users engage more with a change simply because it is new, so early results overstate the long-term lift. The primacy effect is the opposite: existing users resist change at first, so early results understate it. Both fade with time. To detect them, plot the treatment effect by day and by user tenure, compare new users with existing users, and run the test long enough for behaviour to settle.
195
What is an Overall Evaluation Criterion (OEC) in experimentation?
Show answer
An OEC is a single agreed metric, or a weighted combination of metrics, that represents long-term business value and is used to judge experiments. Good OECs are sensitive enough to move within a test's duration yet aligned with long-term goals; for example, sessions per user with penalties for churn rather than raw clicks. Having an OEC prevents teams from each optimising their own metric and makes ship decisions consistent across the organisation.
196
What is Bayesian A/B testing and how are its results reported?
Show answer
Bayesian A/B testing starts with a prior on each variant's metric, updates it with observed data and produces a posterior distribution. Instead of a p-value, you report things like the probability that treatment beats control, say 94 percent, and the expected loss if you choose the wrong variant. Business teams find this easier to interpret. Priors must be chosen sensibly, and continuous monitoring still needs discipline about decision rules.
197
Your test shows a 2 percent lift in conversion, but revenue per user went down. What do you do?
Show answer
Do not ship on conversion alone. Check whether the change shifted buyers to cheaper items, increased discount usage or pulled in low-value orders. Break down revenue into conversion times average order value, look at item mix, and confirm the revenue drop is statistically meaningful rather than noise. Then go back to the agreed primary metric and guardrails; if revenue is the business goal, it should guide the decision.
198
A feature won its A/B test with a 4 percent lift, but after full launch the metric barely moved. Why might that happen?
Show answer
Several reasons are common. Winner's curse: significant results from underpowered tests tend to overestimate the true effect. Novelty effect faded after launch. Seasonality or other launches changed the baseline, so a before-after comparison is misleading. The test population may differ from all users, for example only one platform. Use a holdout after launch to measure real impact, and run tests with adequate power and duration.
199
Your power analysis says you need 200,000 users per group, but the page gets only 5,000 users a week. What are your options?
Show answer
Waiting 80 weeks is unrealistic. Options: test a bolder change so the expected effect is larger; use a more sensitive metric closer to the change, like add-to-cart instead of purchase; apply CUPED to reduce variance; or move the test to a higher-traffic page. If none work, accept a larger MDE explicitly, or decide using qualitative research and a staged rollout with monitoring instead of a formal test.
200
Halfway through a test you discover a bug that broke the treatment for Android users. What do you do?
Show answer
Stop or pause the affected part, fix the bug and restart the test with fresh data, since mixing broken and fixed periods biases results. Do not simply drop Android users from analysis afterwards, because that can break randomisation balance if the bug affected behaviour or logging. Document the incident, and add pre-launch checks such as running the variant through QA on every platform and monitoring guardrails in the first hours.
201
Write Python code to calculate the sample size per group needed to detect a lift from 5 percent to 5.5 percent conversion.
Show answer
from statsmodels.stats.power import NormalIndPower
from statsmodels.stats.proportion import proportion_effectsize
baseline, target = 0.05, 0.055
effect = proportion_effectsize(target, baseline) # Cohen's h
n = NormalIndPower().solve_power(
effect_size=effect, alpha=0.05, power=0.8,
ratio=1.0, alternative='two-sided'
)
print(round(n)) # about 31,000 users per group
202
Write Python code to compare revenue per user between two groups using Welch's t-test.
Show answer
import numpy as np
from scipy import stats
ctrl = df.loc[df.variant == 'control', 'revenue'].to_numpy()
treat = df.loc[df.variant == 'treatment', 'revenue'].to_numpy()
t, p = stats.ttest_ind(treat, ctrl, equal_var=False) # Welch
diff = treat.mean() - ctrl.mean()
se = np.sqrt(treat.var(ddof=1) / len(treat) + ctrl.var(ddof=1) / len(ctrl))
print(f'diff={diff:.2f} 95% CI=({diff - 1.96*se:.2f}, {diff + 1.96*se:.2f}) p={p:.4f}')
203
Tell me about a time an experiment result changed a product decision.
Show answer
Situation: the team was confident a redesigned home page would increase orders. Task: I owned the experiment design and analysis. Action: I set conversion as primary and session length and refunds as guardrails, ran it for two full weeks and found no lift in orders but a drop for older app versions. Result: we fixed the performance issue, retested, and shipped a version that raised orders by about 1.5 percent.
204
Your LLM gives different summaries for the same document on different runs, but auditors need consistent output. What do you do?
Show answer
Set temperature to zero or near zero and fix other sampling settings, and use a seed where the API supports it, though outputs may still vary slightly. Pin the exact model version. Use a structured output format with fixed fields, and reduce ambiguity in the prompt. Cache and store each generated output with its inputs and version, so the stored result, not a regeneration, is what auditors see.
205
Write a Python function to split a long text into overlapping chunks by word count for a RAG index.
Show answer
def chunk_words(text, size=200, overlap=40):
if overlap >= size:
raise ValueError('overlap must be smaller than size')
words = text.split()
chunks, step = [], size - overlap
for start in range(0, len(words), step):
chunks.append(' '.join(words[start:start + size]))
if start + size >= len(words):
break
return chunks
print(len(chunk_words(doc_text)))
206
What are quantile and power transforms such as Box-Cox and Yeo-Johnson, and when do you use them?
Show answer
Power transforms reshape a skewed feature to look more normal. Box-Cox needs strictly positive values, while Yeo-Johnson also handles zero and negative values; both learn a lambda parameter from training data. A quantile transform maps values to a uniform or normal distribution by rank, which is robust to outliers. They help linear models, distance-based methods and neural networks, but tree models rarely need them.
207
Soon after a new model version went live, customer complaints spiked. How do you roll back safely and learn from it?
Show answer
Roll back immediately to the previous registry version through the deployment pipeline, rather than editing servers by hand, and confirm metrics recover. Then investigate offline: compare predictions of both versions on the same traffic, check feature pipelines and data changes, and review what the offline evaluation missed. Add the failing cases to the test set, and use canary or shadow rollout with alert thresholds for future releases.
208
Write Python code that validates prediction requests with Pydantic before passing them to a model.
Show answer
from pydantic import BaseModel, Field, ValidationError
class LoanRequest(BaseModel):
age: int = Field(ge=18, le=100)
income: float = Field(gt=0)
city: str = Field(min_length=2)
def predict(payload: dict):
try:
req = LoanRequest(**payload)
except ValidationError as e:
return {'error': e.errors()}
return {'score': float(model.predict_proba([[req.age, req.income]])[0, 1])}
209
Tell me about a time you automated a manual step in a machine learning workflow.
Show answer
Situation: each monthly retrain needed a day of manual data pulls, notebook runs and copying files to servers. Task: make it reliable and hands-off. Action: I wrapped the steps into a scheduled pipeline with data validation checks, logged runs in MLflow and added an approval step before promotion. Result: retraining time dropped from a day to under an hour of review, and there were no more failed deployments from missed steps.
Frequently asked questions
Who should practise medium Data Science questions?
Anyone with some hands-on practice: most real interview time is spent on medium questions like these.
How should I practise these?
Answer each question aloud before opening the answer, mark it as solved only when your answer matches the key idea, and save misses to Last-Minute Revision.
Other difficulty levels
More ways to practise
Company-wise Data Science questionsPackage-wise Data Science questionsExperience-wise Data Science questionsRole-wise Data Science questionsCity-wise Data Science questionsMost asked Data Science questionsScenario-based questionsPractical / hands-on questionsHR & behavioural questionsData Science mock interviewOnline assessment questionsTechnical round questionsManagerial round questionsData Science topic sheets