Scenario-Based Data Science Interview Questions

Tie metrics to business outcomes such as cost saved, show feature importance or SHAP values, give example predictions and be transparent about limitations.

Use stratified splits, class weights, resampling (SMOTE or undersampling) and threshold tuning, and judge the model with PR-AUC or F1 instead of accuracy.

Read in chunks or use usecols and dtypes, convert strings to category, avoid apply with row loops in favour of vectorised ops, and consider Polars, Dask or Parquet storage.

Define churn and the prediction window, build behavioural features, split by time, train and compare models, evaluate with recall and lift, explain drivers and monitor drift after deployment.

Define churn and the prediction window → build usage and billing features → handle imbalance → train gradient boosting → evaluate with PR-AUC and lift → explain with SHAP → deploy as batch scoring or an API → monitor drift → hand top-risk users to the retention team.

This is overfitting. Check for data leakage, simplify the model, add regularization, get more data or augment it, use cross-validation, early stopping and feature selection.

The warning appears when you assign values to something that may be a copy of a slice instead of the original DataFrame, for example df[df.age > 30]['flag'] = 1. The original may not change. Fix it by using a single .loc call, like df.loc[df.age > 30, 'flag'] = 1, or by taking an explicit .copy() when you really want a separate DataFrame. Copy-on-write in newer Pandas versions makes this behaviour more predictable.

The category dtype stores each unique value once and keeps small integer codes for every row. For a column like city with 50 unique values in 10 million rows, this can cut memory by more than 90 percent and speed up groupby and sorting. It helps when a column has low cardinality. It does not help for unique IDs or free text, and adding new categories later needs extra handling.

First pin the environment: a requirements.txt or conda environment file with exact versions, or a Docker image. Set random seeds for NumPy, scikit-learn and any framework you use. Remove hard-coded local paths and read them from a config file. Make sure data comes from a shared, versioned location. Finally, restart the kernel and run the notebook top to bottom, so hidden cell order bugs are caught before sharing.

Because of the GIL, threads in CPython do not run Python bytecode in parallel. Threads are fine for I/O-bound work like calling APIs or reading many files. For CPU-bound work like feature computation on chunks, multiprocessing or joblib runs separate processes on separate cores. The cost is process start-up time and copying data between processes. Many NumPy and Pandas operations already release the GIL internally, so measure before adding complexity.

The join key is probably not unique on the right side, so each left row matches several right rows and the result multiplies. Check with right['key'].duplicated().sum(). Other causes are null or blank keys matching each other after cleaning, or joining on the wrong column. Fix it by deduplicating or aggregating the right table first, and use merge(..., validate='many_to_one') so Pandas raises an error when the assumption breaks.

Keep notebooks for exploration and reporting only. Move reusable logic into a Python package with modules for data loading, features, training and evaluation, and import it into notebooks. Keep settings in config files, write unit tests for feature functions, and use pre-commit with a formatter and linter. Track experiments in a tool like MLflow. A clear folder template and code reviews keep the team consistent as the project grows.

Each test at 5 percent significance has a 5 percent false positive chance. Run 20 independent tests and you expect about one false positive even if nothing is real. Corrections include Bonferroni, which divides alpha by the number of tests and is strict, and Benjamini-Hochberg, which controls the false discovery rate and keeps more power. Also reduce the problem by choosing one primary metric before the experiment starts.

I would ask them to wait for the planned duration. Checking early and stopping on the first significant result, called peeking, inflates false positives. Two days also misses weekly patterns and novelty effects, where users click a new feature just because it is new. If speed matters, we can use a sequential testing method designed for early stopping. Otherwise we run the full sample size calculated before launch.

Simpson's paradox happens when a trend in several groups reverses when the groups are combined. Example: a new checkout converts better on both mobile and desktop, but worse overall, because most of its traffic came from mobile, which has lower conversion anyway. The mix of groups, not the treatment, drives the total. The lesson is to check segment-level results and confounding variables before drawing conclusions from aggregated numbers.

First rule out data issues: tracking changes, pipeline delays, missing partitions or a changed metric definition. Then compare with the same week last year and look for holidays or seasonality. Check normal weekly variation to see if 8 percent is unusual. Break the drop down by platform, region, app version and user type to find where it is concentrated. Finally, check releases, campaigns or outages that happened that week.

A sample ratio mismatch means the observed split between groups differs from the planned split, for example 52 to 48 instead of 50 to 50 with a large sample. A chi-square test on the counts confirms it. It usually signals a bug in assignment, redirects, bot filtering or logging, so the results cannot be trusted. Do not analyse the metric; find and fix the root cause and rerun the test.

Without randomisation I would use quasi-experimental methods. Difference-in-differences compares the change in a group that got the feature early with one that did not, if rollout was staggered. Interrupted time series or synthetic control builds a counterfactual from pre-launch trends or similar markets. Regression discontinuity works if eligibility had a cut-off. I would state the assumptions clearly and check them with placebo tests.

If only 1 percent of transactions are fraud, a model that predicts not fraud for everything is 99 percent accurate and useless. Accuracy hides the minority class. I would check the confusion matrix, recall on fraud cases and precision-recall AUC. Then fix it with class weights, resampling or a lower decision threshold, and choose the threshold based on the cost of missed fraud versus false alerts.

R-squared shows the share of variance in the target explained by the model. It never decreases when you add a feature, even a useless one. Adjusted R-squared penalises extra features based on the number of predictors and sample size, so it rises only if a new feature improves the model more than chance would. Use adjusted R-squared to compare linear models with different numbers of features.

A learning curve plots training and validation scores against the size of the training set. If both scores are low and close, the model has high bias and needs more features or a more flexible model. If training is high but validation is much lower and the gap narrows as data grows, the model has high variance and more data will help. It tells you whether collecting more data is worth the cost.

Random k-fold mixes future and past rows, so the model trains on data from after the period it predicts. That leaks information and inflates scores. Use time-based splits instead: train on an expanding or rolling window and validate on the next period, like TimeSeriesSplit in scikit-learn. Leave a gap between train and validation when features use lagged values, so the test reflects real forecasting conditions.

The default 0.5 is rarely right. I would get the cost of each outcome: the value of saving a churner, the cost of a retention offer, and the team's capacity to contact customers. Then I compute expected profit or precision and recall at many thresholds on validation data and pick the one that maximises value within capacity. I would also check that the chosen threshold stays stable over recent months.

I would give a clear, individual explanation, not a technical dump. Using SHAP or reason codes, I would list the top factors, such as high existing EMI burden or short credit history, and say what could improve the outcome. I would also check that the model does not rely on protected attributes or close proxies. For regulated lending, a human review path and documented reason codes are essential.

First measure where the time goes. Then reduce trees and maximum depth and check the accuracy loss. Switch to a gradient boosting model, which often needs fewer, smaller trees. Compile the model with tools like ONNX Runtime or Treelite. Cache features or precompute scores for known users. If accuracy allows, a simpler logistic regression may meet the latency budget easily. Always compare options on the same validation set.

Common causes are a learning rate that is too high, exploding gradients, bad input values, and unstable maths like log of zero. I would check the input data for NaN or infinite values, lower the learning rate, add gradient clipping, and make sure the loss uses the stable built-in version with logits. With mixed precision, I would check the loss scaler. Logging gradient norms per step helps find the exact point it breaks.

With so little data, I would use transfer learning from a pretrained model like ResNet or EfficientNet and fine-tune only the top layers first. I would add data augmentation such as flips, crops and colour changes, use dropout and weight decay, and apply early stopping on validation loss. Collecting more labelled images or using a smaller model also helps. I would check for duplicate images across train and validation sets too.

Reduce the batch size and use gradient accumulation to keep the same effective batch. Shorten the maximum sequence length. Turn on mixed precision training with float16 or bfloat16. Use gradient checkpointing, which saves memory by recomputing activations. Freeze some layers or switch to a smaller model, or use parameter-efficient fine-tuning like LoRA. Also make sure no old tensors stay on the GPU between runs.

If weights start too large, activations and gradients explode; if too small, they vanish, and training stalls. If all weights are equal, every neuron learns the same thing. Good schemes keep the variance of activations stable across layers: Xavier or Glorot initialisation for tanh and sigmoid, and He initialisation for ReLU. Frameworks use sensible defaults, but it matters when building custom layers or very deep networks.

First measure the gap with a labelled Hinglish test set. Hinglish mixes scripts and spellings, so tokenisers built for English split it badly. I would try a multilingual or Indic model such as mBERT, XLM-R or IndicBERT, normalise common spellings, and add labelled Hinglish data, possibly with transliteration. Evaluate per language segment and monitor the language mix in production, because it changes over time.

Target encoding replaces each category with the average target value for that category, for example the churn rate per city. It handles high-cardinality columns compactly. The main risk is leakage: if a row's own target is included in its encoding, the model sees the answer and overfits. Prevent it with out-of-fold encoding, smoothing toward the global mean for rare categories, and fitting only on training data.

An ID has no real meaning, so high importance means the model is memorising individual customers. It often happens when the same customers appear in both train and test, or when IDs were assigned in time order and act as a proxy for date or signup cohort. I would drop the ID, use a group-based split by customer, and if timing matters, add an explicit feature like account age instead.

Feature hashing, the hashing trick, maps categories or tokens into a fixed number of columns using a hash function, without storing a vocabulary. It handles very high-cardinality or ever-changing values, like URLs or device IDs, with constant memory and works well in streaming systems. The trade-off is collisions, where different values share a column, and the features cannot be reversed for explanation. Choose a large enough number of buckets.

This is training-serving skew. The feature might be computed from data that arrives later, like final order status. First confirm exactly when each feature is available in production. Rebuild the training set using point-in-time values, as they were at prediction time, or drop the feature. A feature store with point-in-time joins and the same feature code for training and serving prevents this. Expect offline scores to drop, but they become honest.

Every feature must use only data available before the forecast time. Use lags and rolling windows that shift past the current point, and respect the forecast horizon: for a 7-day-ahead forecast, the most recent usable lag is 7 days. Compute scalers and encodings inside each training fold. Use known-future features only when truly known, like holidays or planned promotions. Validate with rolling-origin backtests that mimic production.

Find which columns caused the explosion, usually a few high-cardinality categoricals. Group rare categories into Other, or switch those columns to frequency, target or hashing encoding. Use sparse matrices so zeros take no memory. Tree libraries like LightGBM and CatBoost can handle categoricals natively without one-hot. Then remove near-constant columns and check that validation scores hold up with the smaller feature set.

Impurity-based importance, the default in tree models, sums how much each feature reduces impurity across splits. It is fast but biased toward high-cardinality and continuous features, and it is computed on training data. Permutation importance shuffles one feature at a time on validation data and measures the drop in score, so it reflects real predictive value. Both can understate correlated features, because the model can lean on the copy.

An index is a separate sorted structure, usually a B-tree, that lets the database find rows without scanning the whole table, like a book index. It speeds up filters, joins and sorting on the indexed columns. Indexes cost storage and slow down inserts, updates and deletes because each index must be updated. They may be ignored for low-selectivity columns or when a function is applied to the column in WHERE.

Start with EXPLAIN to see the plan and find full scans, bad joins or spills. Filter as early as possible and select only needed columns. Make sure the table is partitioned or clustered on the date column and that the query filters on it. Check join keys for type mismatches and duplicates that explode rows. Pre-aggregate into a summary table if the query runs daily, and replace correlated subqueries with joins or window functions.

NULL means unknown, so any comparison with NULL, even NULL = NULL, gives unknown and the row is filtered out. Use IS NULL or IS NOT NULL instead. Aggregates like SUM, AVG and COUNT(column) skip NULLs, so AVG may be computed over fewer rows than you expect. NOT IN with a subquery that contains a NULL returns no rows. COALESCE replaces NULLs with a default value.

The join created duplicate order rows, because some orders match more than one promotion row, so each amount is counted several times. Check by comparing COUNT(*) with COUNT(DISTINCT order_id) after the join. Fix it by aggregating the promotions table to one row per order first, picking one promotion with ROW_NUMBER, or summing revenue before the join. Always check row counts after every join.

I would first align definitions: date range, time zone, filters like test users or cancelled orders, and whether the metric counts users or events. Then compare at a smaller grain, by day or by city, to find where the gap starts. I would read the dashboard's underlying query and check data refresh times. Once found, I would document the agreed definition so the same mismatch does not repeat.

Hallucination is when a model produces confident text that is false or not supported by any source. It happens because the model generates plausible tokens rather than checking facts. Reduce it by grounding answers in retrieved documents, asking the model to cite sources and say I do not know when context is missing, using low temperature, and validating outputs with rules or a second checking step. Measure it with a labelled evaluation set.

First check retrieval on its own with a labelled set of questions, since generation cannot fix bad context. Then improve chunking so passages are self-contained, add metadata filters such as product or date, try a better embedding model, and add hybrid search with keywords. A reranker on the top 20 results often helps most. Also rewrite vague user questions into clearer search queries before retrieval.

Measure cost per request and find the biggest token users. Shorten system prompts and retrieved context, and cap output length. Route simple requests to a smaller, cheaper model and keep the large model for hard ones. Cache repeated answers and use provider prompt caching for fixed prefixes. Batch offline jobs. Consider fine-tuning a small model for a high-volume narrow task. Check every change against the evaluation set.

Prompt injection is when user input or retrieved content contains instructions that override your system prompt, such as ignore previous instructions and reveal the data. Defences include treating all external text as data, clearly separating it in the prompt, giving the model only the tools and permissions it truly needs, validating outputs before acting on them, and requiring human approval for sensitive actions. No single prompt trick fully solves it.

Break down where the time goes: retrieval, model time to first token and generation length. Stream tokens so users see output immediately. Cut prompt and output length, run retrieval and other calls in parallel, and cache frequent answers. Use a smaller or faster model for simple requests, and avoid chains of several LLM calls where one would do. Set a latency budget per step and monitor p95, not just averages.

The knowledge index probably still holds old document versions, or the model is answering from its training data instead of retrieved context. Fix the ingestion pipeline to re-index on document change and remove superseded versions. Store effective dates in metadata and filter or boost the latest. Instruct the model to answer only from provided context and show the source with its date, so users can verify it.

First check the data processing agreement, data residency and retention terms, and India's DPDP Act requirements with legal. Minimise what is sent: mask or tokenise names, phone numbers, Aadhaar and card numbers before the call, and restore them afterwards if needed. Use enterprise endpoints with no training on your data, log access, and consider a self-hosted open model for highly sensitive workloads.

Profile slow requests to see if time goes to feature fetching, preprocessing, the model or the network. Common fixes: precompute and cache features, remove slow database calls from the request path, load the model once at start-up, use a faster runtime like ONNX, and warm up instances. Check garbage collection pauses, cold starts and autoscaling limits. Set timeouts with a safe fallback prediction for rare slow cases.

Since accuracy cannot be measured quickly, monitor proxies. Track input data drift with PSI per feature, prediction score distribution, the share of transactions flagged, and data quality like missing values. Watch early signals such as chargeback complaints or manual review outcomes on a sample. Set alerts on sudden shifts. When labels arrive, compute real precision and recall by cohort and compare them with the proxy alerts to tune them.

Scheduled retraining, like monthly, is simple and predictable but may be too late after a sudden change or wasteful when nothing changed. Trigger-based retraining runs when monitoring shows drift or a performance drop, which reacts faster but needs reliable monitoring. Many teams combine both: a regular schedule plus triggers for big shifts. Either way, a retrained model must pass the same evaluation gate before replacing the current one.

Usually preprocessing differs between training and serving. Check that the same pipeline object was saved, not just the estimator, and that category order, missing value handling and scaling match. Compare library versions, feature types like strings versus integers, column order, and time zones in date features. Send the same few rows through both paths and compare features step by step to find the first mismatch.

I would start from the team's pain points, then provide shared building blocks: a standard project template, managed notebooks with access to the warehouse, experiment tracking and a model registry, an orchestrator like Airflow for pipelines, a feature store if real-time use cases exist, a standard serving path using containers, and default monitoring. Prefer managed services over custom builds, and add self-serve documentation so teams do not depend on one platform engineer.

The model is treating Diwali as noise because the festival moves on the calendar and a fixed yearly seasonality does not align. Add explicit holiday features: a flag for Diwali and a few days before and after, since buying starts early. Include other regional festivals and sale events too. Models like Prophet or gradient boosting with holiday features handle this well. Then backtest specifically on past festive periods to confirm the fix.

Three weeks is too little to learn seasonality, so borrow information. Find similar past launches by category, price band and channel, and use their early sales curves as analogues. A global model trained across many products, with product attributes as features, can also forecast a cold-start item. Combine this with planned marketing spend and business judgement, show a wide interval, and update the forecast weekly as real data arrives.

This is intermittent demand, and standard models like ARIMA forecast a small constant that is useless for stocking. Croston's method and its SBA variant forecast the size of non-zero demand and the interval between sales separately. Alternatives are aggregating to weekly level, or using a global gradient boosting model with Tweedie or Poisson loss. Evaluate with MAE or MASE, not MAPE, because zero actuals break percentage errors.

First make the gaps explicit by reindexing to a full daily calendar so missing days appear as NaN instead of silently disappearing. For short gaps, linear or time-based interpolation is usually fine. If there is strong weekly seasonality, fill using the same weekday from neighbouring weeks or a seasonal decomposition. Add a flag column marking imputed days, and never treat missing as zero unless zero really happened.

In backtesting you used the actual temperature for future days, which you will never know at forecast time. In production the model gets a weather forecast, which has its own error, so performance drops. This is a subtle form of leakage. Fix the backtest by using the weather forecast that was available on each forecast date, or lagged actuals, so offline results match what production can really achieve.

One classical model per SKU is simple and interpretable but expensive to maintain, and it fails for short or sparse series. A global model, such as LightGBM trained on all SKUs with lags, calendar and product features, learns shared patterns and handles new items better. In practice I would use a global model as the main engine, keep statistical models for a few high-value series, and compare both in a rolling backtest.

That period is a structural break, not normal behaviour, and letting a model learn from it raw will distort trend and seasonality. Options are to add an indicator feature so the model attributes the drop to the event, to replace the period with values from a counterfactual estimate, or to train only on data after the recovery if enough history exists. Compare options with backtests on recent normal periods and document the choice.

A forecast of the mean is the wrong target when costs are asymmetric. Forecast a quantile instead. The optimal quantile here is about 5 divided by 5 plus 1, roughly the 83rd percentile, which is the newsvendor critical ratio. Train with quantile or pinball loss, or use prediction intervals from a probabilistic model. Evaluate with pinball loss and the actual stockout and holding cost, not just MAE.

An average hides the shape of the distribution. Delivery times are usually right-skewed, so a 30-minute mean can include many orders taking 60 minutes or more. Look at the median, the 90th and 95th percentiles and a histogram, then break them down by city, time of day and restaurant. Report the share of orders beyond the promised time, because that is what customers actually experience.

First confirm with per-segment metrics and enough samples, since small groups are noisy. Then check why: fewer training examples from that region, different feature distributions, missing data or labels that reflect past biased decisions. Fixes include collecting more data, reweighting, segment-aware features or a separate calibration. Also review fairness and regulatory requirements, and agree with risk and compliance teams on acceptable performance gaps before launch.

With so few defects, a plain classifier will learn to say no defect. Collect every defect image possible and oversample them or use class-weighted or focal loss. Consider anomaly detection trained only on good parts, which flags anything unusual. Use augmentation that mimics real variation in lighting and position. Evaluate with recall at a fixed false alarm rate, since missing a defect usually costs far more than a manual recheck.

This is domain shift: user photos have clutter, blur, poor lighting and odd angles that the model never saw. Collect and label a sample of real user photos and evaluate on them separately. Add augmentation for blur, noise, colour jitter, rotation and random crops, and fine-tune on a mix of studio and real images. Monitor accuracy on user photos in production, since that is the distribution that matters.

The model has learned a shortcut: the watermark correlates with the label in your dataset, perhaps because images of one class came from a single source. It will fail when that correlation breaks. Remove or crop the watermarks, rebalance sources across classes, and create a validation set from different sources. Keep using saliency maps and slice-based evaluation to catch other spurious cues such as backgrounds or text.

First measure agreement, for example with Cohen's kappa or IoU between annotators' boxes, to find which classes are confusing. Usually the guidelines are unclear, so write precise rules with example images for edge cases and retrain annotators. Have multiple people label a subset and resolve conflicts by majority vote or an expert reviewer. Track ambiguous images separately, since even a perfect model cannot beat human label noise.

Fixed templates will not scale to hundreds of layouts. Start with preprocessing: deskew, denoise and detect text regions. Run OCR to get words with their positions. Then use a layout-aware document model, or a vision-language model, to map words to fields like invoice number, GSTIN and total. Add validation rules, such as GSTIN format and line items summing to the total, and send low-confidence documents to human review.

Small objects lose detail after downsampling. Train and infer at higher input resolution, or tile large images into overlapping crops. Use a backbone with a feature pyramid so detection happens on high-resolution maps. Tune anchors or assignment settings for small sizes, and oversample images with small objects. Check labelling quality for tiny boxes too. Measure mAP on small objects separately to confirm improvement.

Start with a lightweight one-stage detector at the smallest input size that keeps acceptable accuracy. Optimise it: export to ONNX, then use a hardware runtime like TensorRT or OpenVINO, with FP16 or INT8 quantisation calibrated on real images. Prune or distil from a larger model if needed. Profile the full pipeline, including decoding and preprocessing, and run detection every few frames with tracking in between if compute is tight.

Do not pick 3,000 at random. Label a small diverse seed set, using embedding clustering to cover variety, and train a first model. Then use active learning: label the images the model is least confident about or that disagree across models. Use semi-supervised methods like pseudo-labelling high-confidence images, or self-supervised pretraining on all 50,000. Keep a randomly sampled test set so evaluation stays unbiased.

Start with a pretrained sentence embedding model plus a simple classifier, or fine-tune a small transformer with class weights. For the rare categories, try few-shot approaches such as SetFit or an LLM with examples in the prompt, and generate or collect more examples. Merge categories that are rarely distinguishable. Report per-class F1 rather than overall accuracy, and route low-confidence predictions to a human queue.

Do not ship on conversion alone. Check whether the change shifted buyers to cheaper items, increased discount usage or pulled in low-value orders. Break down revenue into conversion times average order value, look at item mix, and confirm the revenue drop is statistically meaningful rather than noise. Then go back to the agreed primary metric and guardrails; if revenue is the business goal, it should guide the decision.

Not significant does not mean no effect. Report the observed effect with its confidence interval, for example minus 0.3 to plus 1.2 percent, and compare it with the MDE the test was powered for. If the interval rules out effects that matter, conclude the change has little impact and decide based on cost or design preference. If the interval is wide, the test was underpowered, and a longer test or variance reduction may help.

Run an A/B/n test with control plus five variants and equal random allocation. Calculate sample size for the smallest effect worth detecting, remembering that each variant gets only a sixth of traffic, so the test takes longer. Plan the analysis upfront: compare each variant to control and adjust for multiple comparisons. If the team has strong priors, test fewer designs first to save time.

Several reasons are common. Winner's curse: significant results from underpowered tests tend to overestimate the true effect. Novelty effect faded after launch. Seasonality or other launches changed the baseline, so a before-after comparison is misleading. The test population may differ from all users, for example only one platform. Use a holdout after launch to measure real impact, and run tests with adequate power and duration.

Waiting 80 weeks is unrealistic. Options: test a bolder change so the expected effect is larger; use a more sensitive metric closer to the change, like add-to-cart instead of purchase; apply CUPED to reduce variance; or move the test to a higher-traffic page. If none work, accept a larger MDE explicitly, or decide using qualitative research and a staged rollout with monitoring instead of a formal test.

Stop or pause the affected part, fix the bug and restart the test with fresh data, since mixing broken and fixed periods biases results. Do not simply drop Android users from analysis afterwards, because that can break randomisation balance if the bug affected behaviour or logging. Document the incident, and add pre-launch checks such as running the variant through QA on every platform and monitoring guardrails in the first hours.

Treated and control buyers compete for the same sellers' supply, so a treatment that makes treated buyers buy more takes inventory from control buyers, exaggerating the measured lift. Use designs that limit interference: switchback tests that alternate the whole market between variants over time slots, or randomise by city or region. Analyse at the switch or cluster level with appropriate variance, and accept that these tests need longer to reach power.

Plan a geo experiment: select comparable cities or regions, randomly assign some to run the campaign and keep others as control, matching them on past sales trends. Measure the difference in sales during and after the campaign, using methods like difference-in-differences or synthetic control to build a counterfactual. Run a pre-period check that test and control regions track each other, and include enough regions for reasonable power.

Likely causes are late-arriving events, deduplication running after your query, time zone boundaries or records being updated. First compare runs to see which rows change. Then define the metric on a stable basis: use event time with a fixed cut-off, wait for a data freshness window before publishing, and snapshot the result. Document whether numbers are provisional, and make pipelines idempotent so reruns give the same answer.

Set temperature to zero or near zero and fix other sampling settings, and use a seed where the API supports it, though outputs may still vary slightly. Pin the exact model version. Use a structured output format with fixed fields, and reduce ambiguity in the prompt. Cache and store each generated output with its inputs and version, so the stored result, not a regeneration, is what auditors see.

Depending on the encoder, the pipeline may crash or silently map the value to all zeros. Plan for it: use OneHotEncoder with handle_unknown='ignore' or map unseen values to an explicit 'other' bucket that also exists in training for rare values. Log and monitor the rate of unknown categories, since a rising rate signals data drift, and retrain when new categories become common.

Roll back immediately to the previous registry version through the deployment pipeline, rather than editing servers by hand, and confirm metrics recover. Then investigate offline: compare predictions of both versions on the same traffic, check feature pipelines and data changes, and review what the offline evaluation missed. Add the failing cases to the test set, and use canary or shadow rollout with alert thresholds for future releases.

Profile utilisation first; idle or under-filled GPUs are common. Enable dynamic batching so each GPU call handles several requests, right-size instance types and autoscale on queue depth rather than CPU. Reduce model cost with quantisation, distillation or a smaller model for easy requests. Cache repeated predictions, move non-urgent work to batch jobs on cheaper hardware, and track cost per thousand predictions as a standing metric.