Data Science Interview Questions for Architects & Directors (₹40–50 LPA)

Splitting text into units such as words, subwords or characters that are mapped to ids; subword methods like BPE handle rare words well.

Information from outside the training window, such as future data or test statistics, enters training and inflates performance. Split first, then fit scalers and encoders on training data only.

A network that compresses inputs into a small latent code and reconstructs them; used for denoising, dimensionality reduction and anomaly detection.

An iterable can produce an iterator (it has __iter__); an iterator has __next__ and remembers its position. A for-loop calls iter() on the iterable and then next() repeatedly.

Gradients shrink as they pass through many layers (for example with sigmoid), so early layers barely learn. ReLU, residual connections and good initialisation help.

Pearson measures linear relationships on continuous data; Spearman uses ranks, so it captures monotonic relationships and is less sensitive to outliers.

Define the hypothesis and one primary metric, calculate sample size from the baseline, minimum detectable effect, α and power, randomise users, run for full business cycles, check for sample-ratio mismatch, then test both statistical and practical significance.

CNNs use convolutions to find spatial patterns (images). RNNs/LSTMs process sequences step by step but struggle with long context. Transformers use self-attention to look at all tokens at once — parallel and better at long-range context; they power modern LLMs.

SVM finds the maximum-margin separating hyperplane. Kernels such as RBF compute similarity in a higher-dimensional space implicitly, so non-linear boundaries are possible.

Start with a sensible default such as 1e-3 for Adam, run a learning-rate range test, and use schedulers or warm-up while watching the loss curve.

Correlation measures linear association between variables (-1 to 1). A third factor, reverse causality or chance can create it, so causation needs controlled experiments.

Transformers use self-attention to process all tokens in parallel and capture long-range context, making them faster to train and more scalable than sequential RNNs.

Regression predicts continuous values such as price; classification predicts discrete categories such as spam or not spam.

The distribution of sample means approaches a normal distribution as the sample size grows (roughly 30+), whatever the population's shape. It lets us build confidence intervals and run hypothesis tests on means.

An optimisation method that updates weights in the opposite direction of the loss gradient. The learning rate sets the step size: too high and it diverges, too low and it is slow. Variants include SGD, mini-batch and Adam.

The ROC curve plots true positive rate against false positive rate across thresholds. AUC is the probability that a random positive is ranked above a random negative; 0.5 is chance.

loc selects by label (row and column names, inclusive of the end label); iloc selects by integer position (end-exclusive).

ReLU for hidden layers, sigmoid for binary outputs, softmax for multi-class outputs and tanh when zero-centred outputs help; Leaky ReLU avoids dead neurons.

The probability of seeing a result at least as extreme as the observed one if the null hypothesis were true. A small p-value is evidence against the null, not proof that the alternative is true.

Modelling the count of events in a fixed interval when events occur independently at a constant average rate, such as calls per hour or defects per batch.

It computes the gradient of the loss with respect to every weight by applying the chain rule backwards through the network, so gradient descent can update them.

A mutex in CPython that lets only one thread execute Python bytecode at a time. Use multiprocessing for CPU-bound work and threads or asyncio for I/O-bound work.

Train fits the model, validation tunes hyperparameters and model choice, and the untouched test set gives a final unbiased estimate of performance.

It tries candidate splits and picks the one that most reduces impurity — Gini or entropy (information gain) for classification, variance / MSE for regression.

The probability of seeing a result at least this extreme if the null hypothesis were true. A small p-value (for example < 0.05) is evidence against the null hypothesis.

It models the log-odds of a binary outcome as a linear function and passes it through a sigmoid to give probabilities, trained by maximising likelihood.

Measure first with cProfile or timeit, then replace loops with NumPy or Pandas vectorised calls, use built-ins, caching and better data structures.

Tie metrics to business outcomes such as cost saved, show feature importance or SHAP values, give example predictions and be transparent about limitations.

The rule that lets arrays of different shapes operate together by virtually stretching the smaller one, for example adding a scalar or a (3,) vector to a (4,3) matrix.

Bias occurs when the sample does not represent the population. Reduce it with random or stratified sampling, higher response rates and by checking sample demographics against the population.

Retrieval-Augmented Generation fetches relevant documents and gives them to an LLM as context, reducing hallucination and keeping answers grounded in current private data.

Training a meta-model on the predictions of several base models, often using cross-validated predictions to avoid leakage.

A shallow copy duplicates the outer object but shares nested objects. A deep copy (copy.deepcopy) recursively duplicates everything, so changes to nested items do not leak.

A function using yield that produces values lazily, one at a time. It saves memory when processing large files or streams because the full sequence is never held in RAM.

A method is a function that belongs to an object or class and receives the instance (self) as its first argument.

Accuracy, F1 and confusion matrices for classification, BLEU or ROUGE for generation and summarisation, perplexity for language models, plus human review.

*args collects extra positional arguments into a tuple; **kwargs collects extra keyword arguments into a dict. They make function signatures flexible and are common in wrappers and decorators.

Discriminative models learn P(y given x) directly (logistic regression, SVM). Generative models learn the joint P(x, y) and can generate data (Naive Bayes, GANs).

It applies Bayes' theorem assuming features are independent given the class. The assumption is rarely true but the model is fast and works well for text.

A decorator wraps a function to add behaviour such as logging, timing or caching without changing its code. It is applied with the @name syntax above the function.

It projects data onto orthogonal directions of maximum variance to reduce dimensions, remove correlation and speed up models, at some cost of interpretability.

It fits a line minimising squared error. Assumes linearity, independent errors, constant variance (homoscedasticity), normally distributed residuals and no strong multicollinearity.

Parametric tests assume a distribution (usually normal) and use means; non-parametric tests such as Mann-Whitney use ranks and make fewer assumptions.

A forest averages many trees trained on bootstrapped rows and random feature subsets, which cuts variance and overfitting at the cost of interpretability.

Convolution filters slide over the input to detect local patterns, pooling reduces size, and deeper layers combine edges into shapes and objects. Used mainly for images.

Both measure node impurity; Gini is cheaper to compute and entropy is slightly more sensitive to class balance, with similar trees in practice.

Add a classification head, freeze or gradually unfreeze layers, use a small learning rate, train on labelled data with validation, and track F1 per class.

pd.pivot_table(df, index='region', columns='month', values='sales', aggfunc='sum') reshapes long data into a summary matrix; melt() does the reverse.

This is overfitting. Check for data leakage, simplify the model, add regularization, get more data or augment it, use cross-validation, early stopping and feature selection.

Mean is the average, median is the middle value after sorting and mode is the most frequent value. Median is more robust to outliers.

A symmetric bell-shaped distribution defined by mean and standard deviation, where about 68%, 95% and 99.7% of values lie within 1, 2 and 3 standard deviations.

Use grid search, random search or Bayesian optimisation with cross-validation, starting with a coarse random search and narrowing the promising ranges.

Use stratified splits, class weights, resampling (SMOTE or undersampling) and threshold tuning, and judge the model with PR-AUC or F1 instead of accuracy.

Supervised learning trains on labelled data to predict a target; unsupervised learning finds structure such as clusters or components in unlabelled data.

Small anonymous one-line functions, handy as arguments to sort, map, filter or Pandas apply.

Predictors that are highly correlated make coefficients unstable. Detect with VIF above about 5-10, then drop or combine features, use PCA or apply regularisation.

df.drop_duplicates(subset=[...], keep='first') removes repeated rows; use duplicated() first to inspect what will be dropped.

Use the IQR rule (below Q1-1.5 IQR or above Q3+1.5 IQR), z-scores beyond 3, box plots or model-based methods such as Isolation Forest, then decide whether to cap, remove or keep them.

It combines gradient boosting with regularisation, handling of missing values, parallel tree construction and speed optimisations, and it performs strongly on tabular data.

Frequentists treat parameters as fixed and use long-run frequencies; Bayesians treat parameters as distributions updated with data via priors and posteriors, giving direct probability statements.