Data Science Interview Questions for Senior Specialists (₹15–20 LPA)

The square root of the variance; it measures how far values typically sit from the mean, in the same units as the data.

Supervised learning trains on labelled data to predict a target; unsupervised learning finds structure such as clusters or components in unlabelled data.

Transformers use self-attention to process all tokens in parallel and capture long-range context, making them faster to train and more scalable than sequential RNNs.

A method is a function that belongs to an object or class and receives the instance (self) as its first argument.

Correlation measures linear association between variables (-1 to 1). A third factor, reverse causality or chance can create it, so causation needs controlled experiments.

Bias is error from overly simple assumptions (underfitting). Variance is sensitivity to the training data (overfitting). More model complexity lowers bias but raises variance — aim for the lowest total error using cross-validation and regularization.

Split the data into k folds, train on k−1 folds and validate on the remaining one, rotate and average the scores. It gives a more reliable estimate than one split and helps tune hyperparameters without touching the test set.

An iterable can produce an iterator (it has __iter__); an iterator has __next__ and remembers its position. A for-loop calls iter() on the iterable and then next() repeatedly.

Predictors that are highly correlated make coefficients unstable. Detect with VIF above about 5-10, then drop or combine features, use PCA or apply regularisation.

Use the IQR rule (below Q1-1.5 IQR or above Q3+1.5 IQR), z-scores beyond 3, box plots or model-based methods such as Isolation Forest, then decide whether to cap, remove or keep them.

Add a classification head, freeze or gradually unfreeze layers, use a small learning rate, train on labelled data with validation, and track F1 per class.

A mutex in CPython that lets only one thread execute Python bytecode at a time. Use multiprocessing for CPU-bound work and threads or asyncio for I/O-bound work.

df.drop_duplicates(subset=[...], keep='first') removes repeated rows; use duplicated() first to inspect what will be dropped.

A symmetric bell-shaped distribution defined by mean and standard deviation, where about 68%, 95% and 99.7% of values lie within 1, 2 and 3 standard deviations.

Training a meta-model on the predictions of several base models, often using cross-validated predictions to avoid leakage.

Splitting text into units such as words, subwords or characters that are mapped to ids; subword methods like BPE handle rare words well.

A decorator wraps a function to add behaviour such as logging, timing or caching without changing its code. It is applied with the @name syntax above the function.

A forest averages many trees trained on bootstrapped rows and random feature subsets, which cuts variance and overfitting at the cost of interpretability.

A network that compresses inputs into a small latent code and reconstructs them; used for denoising, dimensionality reduction and anomaly detection.

Frequentists treat parameters as fixed and use long-run frequencies; Bayesians treat parameters as distributions updated with data via priors and posteriors, giving direct probability statements.

The probability of seeing a result at least as extreme as the observed one if the null hypothesis were true. A small p-value is evidence against the null, not proof that the alternative is true.

A set stores unique values with no keys; a dictionary stores key-value pairs with unique keys. Both use hashing, so lookups are O(1) on average.

Use stratified splits, class weights, resampling such as SMOTE, threshold tuning and judge models with precision, recall, F1 or PR-AUC instead of accuracy.

CNNs use convolutions to find spatial patterns (images). RNNs/LSTMs process sequences step by step but struggle with long context. Transformers use self-attention to look at all tokens at once — parallel and better at long-range context; they power modern LLMs.

Tie metrics to business outcomes such as cost saved, show feature importance or SHAP values, give example predictions and be transparent about limitations.

It normalises layer inputs per mini-batch and then rescales them, stabilising and speeding up training and allowing higher learning rates.

Small anonymous one-line functions, handy as arguments to sort, map, filter or Pandas apply.

Define churn and the prediction window → build usage and billing features → handle imbalance → train gradient boosting → evaluate with PR-AUC and lift → explain with SHAP → deploy as batch scoring or an API → monitor drift → hand top-risk users to the retention team.

Putting features on comparable ranges using standardisation or min-max scaling. Distance-based and gradient-based models (KNN, SVM, regression, neural nets) need it; tree models do not.

loc selects by label (row and column names, inclusive of the end label); iloc selects by integer position (end-exclusive).

RNNs process sequences with a hidden state but struggle with long dependencies. LSTMs add gates that control what to remember or forget, easing the vanishing gradient.

The model memorises noise and fails on new data. Prevent it with more data, simpler models, regularisation, cross-validation, early stopping and dropout.

The probability of seeing a result at least this extreme if the null hypothesis were true. A small p-value (for example < 0.05) is evidence against the null hypothesis.

A table of true positives, false positives, true negatives and false negatives from which accuracy, precision, recall and F1 are calculated.

It applies Bayes' theorem assuming features are independent given the class. The assumption is rarely true but the model is fast and works well for text.

The distribution of sample means approaches a normal distribution as the sample size grows (roughly 30+), whatever the population's shape. It lets us build confidence intervals and run hypothesis tests on means.

Mean is the average, median is the middle value after sorting and mode is the most frequent value. Median is more robust to outliers.

It lets a model weigh which input positions matter most for each output, computing weighted combinations of values from query-key similarity. It is the core of transformers.

pd.pivot_table(df, index='region', columns='month', values='sales', aggfunc='sum') reshapes long data into a summary matrix; melt() does the reverse.

Use stratified splits, class weights, resampling (SMOTE or undersampling) and threshold tuning, and judge the model with PR-AUC or F1 instead of accuracy.

SVM finds the maximum-margin separating hyperplane. Kernels such as RBF compute similarity in a higher-dimensional space implicitly, so non-linear boundaries are possible.

Accuracy, F1 and confusion matrices for classification, BLEU or ROUGE for generation and summarisation, perplexity for language models, plus human review.

Activations add non-linearity so networks can learn complex patterns. ReLU (max(0, x)) is cheap and reduces vanishing gradients compared with sigmoid/tanh; Leaky ReLU avoids 'dead' neurons.

int, float, complex, bool, str, list, tuple, set, dict and NoneType. Mutable ones are list, set and dict; the others are immutable.

Regularization adds a penalty on weights to reduce overfitting. L1 (Lasso) adds |w| and can push features to exactly zero. L2 (Ridge) adds w² and shrinks weights smoothly.

A shallow copy duplicates the outer object but shares nested objects. A deep copy (copy.deepcopy) recursively duplicates everything, so changes to nested items do not leak.

Define churn and the prediction window, build behavioural features, split by time, train and compare models, evaluate with recall and lift, explain drivers and monitor drift after deployment.

Calibration checks whether predicted probabilities match real frequencies (e.g. 70% predictions are right about 70% of the time). Fix with Platt scaling or isotonic regression.

First understand why data is missing. Then drop rows or columns with too many nulls, impute with mean/median/mode or a model (KNN), add a 'was_missing' flag, and fit imputers on training data only.

Term Frequency × Inverse Document Frequency: a word scores high if it is frequent in one document but rare across all documents. A classic way to turn text into numeric features.

It models the log-odds of a binary outcome as a linear function and passes it through a sigmoid to give probabilities, trained by maximising likelihood.

Gradients shrink as they pass through many layers (for example with sigmoid), so early layers barely learn. ReLU, residual connections and good initialisation help.

The rule that lets arrays of different shapes operate together by virtually stretching the smaller one, for example adding a scalar or a (3,) vector to a (4,3) matrix.

Use grid search, random search or Bayesian optimisation with cross-validation, starting with a coarse random search and narrowing the promising ranges.

Regression predicts continuous values such as price; classification predicts discrete categories such as spam or not spam.

P(A given B) = P(B given A) x P(A) / P(B). In a medical test it converts test accuracy and disease prevalence into the real chance a positive result means disease.

Covariance shows the direction of a relationship but depends on units. Correlation is covariance scaled to −1…+1, so it shows direction and strength and is unit-free.

Modelling the count of events in a fixed interval when events occur independently at a constant average rate, such as calls per hour or defects per batch.

Read in chunks or use usecols and dtypes, convert strings to category, avoid apply with row loops in favour of vectorised ops, and consider Polars, Dask or Parquet storage.

map works element-wise on a Series, apply works on a Series or along a DataFrame axis, and applymap (now DataFrame.map) applies a function to every cell.

This is overfitting. Check for data leakage, simplify the model, add regularization, get more data or augment it, use cross-validation, early stopping and feature selection.

Reusing a model pretrained on a large dataset and fine-tuning it on a smaller task-specific dataset, saving data, time and compute.