Data Science Interview Questions for Experienced Professionals (₹10–12 LPA)

Predictors that are highly correlated make coefficients unstable. Detect with VIF above about 5-10, then drop or combine features, use PCA or apply regularisation.

Putting features on comparable ranges using standardisation or min-max scaling. Distance-based and gradient-based models (KNN, SVM, regression, neural nets) need it; tree models do not.

Define churn and the prediction window → build usage and billing features → handle imbalance → train gradient boosting → evaluate with PR-AUC and lift → explain with SHAP → deploy as batch scoring or an API → monitor drift → hand top-risk users to the retention team.

L1 (Lasso) adds absolute weights and can shrink some to zero, giving feature selection. L2 (Ridge) adds squared weights and shrinks them smoothly without removing features.

Bias is error from overly simple assumptions (underfitting). Variance is sensitivity to the training data (overfitting). More model complexity lowers bias but raises variance — aim for the lowest total error using cross-validation and regularization.

Retrieval-Augmented Generation fetches relevant documents and gives them to an LLM as context, reducing hallucination and keeping answers grounded in current private data.

CNNs use convolutions to find spatial patterns (images). RNNs/LSTMs process sequences step by step but struggle with long context. Transformers use self-attention to look at all tokens at once — parallel and better at long-range context; they power modern LLMs.

Add a classification head, freeze or gradually unfreeze layers, use a small learning rate, train on labelled data with validation, and track F1 per class.

Calibration checks whether predicted probabilities match real frequencies (e.g. 70% predictions are right about 70% of the time). Fix with Platt scaling or isotonic regression.

A decorator wraps a function to add behaviour such as logging, timing or caching without changing its code. It is applied with the @name syntax above the function.

A function using yield that produces values lazily, one at a time. It saves memory when processing large files or streams because the full sequence is never held in RAM.

Split the data into k folds, train on k−1 folds and validate on the remaining one, rotate and average the scores. It gives a more reliable estimate than one split and helps tune hyperparameters without touching the test set.

A measure of asymmetry. Right-skewed data has a long tail on the right (mean greater than median); left-skewed is the opposite. Log transforms often reduce right skew.

Type I = false positive (rejecting a true null), probability α. Type II = false negative (missing a real effect), probability β. Power = 1 − β.

They build a list in one readable expression, for example [x*x for x in data if x > 0]. They are usually faster than an equivalent for-loop with append.

Use grid search, random search or Bayesian optimisation with cross-validation, starting with a coarse random search and narrowing the promising ranges.

Frequentists treat parameters as fixed and use long-run frequencies; Bayesians treat parameters as distributions updated with data via priors and posteriors, giving direct probability statements.

An optimisation method that updates weights in the opposite direction of the loss gradient. The learning rate sets the step size: too high and it diverges, too low and it is slow. Variants include SGD, mini-batch and Adam.

A set stores unique values with no keys; a dictionary stores key-value pairs with unique keys. Both use hashing, so lookups are O(1) on average.

It assigns points to the nearest centroid and updates centroids until stable. Choose k with the elbow method, silhouette score or business context.

It normalises layer inputs per mini-batch and then rescales them, stabilising and speeding up training and allowing higher learning rates.

Use stratified splits, class weights, resampling such as SMOTE, threshold tuning and judge models with precision, recall, F1 or PR-AUC instead of accuracy.

Precision = TP / (TP + FP). Recall = TP / (TP + FN). Prefer recall when missing a positive is costly — for example disease screening or fraud detection.

Transformers use self-attention to process all tokens in parallel and capture long-range context, making them faster to train and more scalable than sequential RNNs.

Parametric tests assume a distribution (usually normal) and use means; non-parametric tests such as Mann-Whitney use ranks and make fewer assumptions.

Monitor input distributions and live metrics, compare to training baselines, and retrain or recalibrate on fresh data when performance drops.

map works element-wise on a Series, apply works on a Series or along a DataFrame axis, and applymap (now DataFrame.map) applies a function to every cell.

Power is the probability of detecting a real effect. Increase it with a larger sample, a bigger effect size, lower noise or a higher significance level.

Regression predicts continuous values such as price; classification predicts discrete categories such as spam or not spam.

Define the hypothesis and one primary metric, calculate sample size from the baseline, minimum detectable effect, α and power, randomise users, run for full business cycles, check for sample-ratio mismatch, then test both statistical and practical significance.

List: ordered and mutable. Tuple: ordered and immutable (hashable, slightly faster). Set: unordered, unique items, fast membership checks. Dict: key–value pairs with fast lookup by key.

Use stratified splits, class weights, resampling (SMOTE or undersampling) and threshold tuning, and judge the model with PR-AUC or F1 instead of accuracy.

It predicts from the k nearest labelled points. It is simple but slow at prediction time, sensitive to scaling and irrelevant features, and weak in high dimensions.

First understand why data is missing. Then drop rows or columns with too many nulls, impute with mean/median/mode or a model (KNN), add a 'was_missing' flag, and fit imputers on training data only.

Regularization adds a penalty on weights to reduce overfitting. L1 (Lasso) adds |w| and can push features to exactly zero. L2 (Ridge) adds w² and shrinks weights smoothly.

A network that compresses inputs into a small latent code and reconstructs them; used for denoising, dimensionality reduction and anomaly detection.

Information from outside the training window, such as future data or test statistics, enters training and inflates performance. Split first, then fit scalers and encoders on training data only.

A forest averages many trees trained on bootstrapped rows and random feature subsets, which cuts variance and overfitting at the cost of interpretability.

Splitting text into units such as words, subwords or characters that are mapped to ids; subword methods like BPE handle rare words well.

int, float, complex, bool, str, list, tuple, set, dict and NoneType. Mutable ones are list, set and dict; the others are immutable.

Gradients shrink as they pass through many layers (for example with sigmoid), so early layers barely learn. ReLU, residual connections and good initialisation help.

Supervised learning trains on labelled data to predict a target; unsupervised learning finds structure such as clusters or components in unlabelled data.

It tries candidate splits and picks the one that most reduces impurity — Gini or entropy (information gain) for classification, variance / MSE for regression.

It projects data onto orthogonal directions of maximum variance to reduce dimensions, remove correlation and speed up models, at some cost of interpretability.

*args collects extra positional arguments into a tuple; **kwargs collects extra keyword arguments into a dict. They make function signatures flexible and are common in wrappers and decorators.

Accuracy, F1 and confusion matrices for classification, BLEU or ROUGE for generation and summarisation, perplexity for language models, plus human review.

ReLU for hidden layers, sigmoid for binary outputs, softmax for multi-class outputs and tanh when zero-centred outputs help; Leaky ReLU avoids dead neurons.

The rule that lets arrays of different shapes operate together by virtually stretching the smaller one, for example adding a scalar or a (3,) vector to a (4,3) matrix.

merge joins on columns, SQL-style (how='inner', 'left', ...). join joins on the index. concat stacks DataFrames along rows or columns.

It applies Bayes' theorem assuming features are independent given the class. The assumption is rarely true but the model is fast and works well for text.

The distribution of sample means approaches a normal distribution as the sample size grows (roughly 30+), whatever the population's shape. It lets us build confidence intervals and run hypothesis tests on means.

Layers of connected neurons that apply weights, biases and activation functions to learn complex patterns, trained by backpropagation.

pd.pivot_table(df, index='region', columns='month', values='sales', aggfunc='sum') reshapes long data into a summary matrix; melt() does the reverse.

Use filter methods (correlation, chi-square), wrapper methods (RFE), embedded methods (Lasso, tree importance) and domain knowledge, validating each choice with cross-validation.

A range built from sample data that would contain the true parameter in a stated percentage (for example 95%) of repeated samples. It is not the probability that the parameter lies inside this one interval.

== compares values for equality, while is checks whether two names point to the same object in memory. Use is mainly for None checks.

It lets a model weigh which input positions matter most for each output, computing weighted combinations of values from query-key similarity. It is the core of transformers.

The ROC curve plots true positive rate against false positive rate across thresholds. AUC is the probability that a random positive is ranked above a random negative; 0.5 is chance.