HCLTech Data Science Interview Questions & Answers

pd.pivot_table(df, index='region', columns='month', values='sales', aggfunc='sum') reshapes long data into a summary matrix; melt() does the reverse.

merge joins on columns, SQL-style (how='inner', 'left', ...). join joins on the index. concat stacks DataFrames along rows or columns.

int, float, complex, bool, str, list, tuple, set, dict and NoneType. Mutable ones are list, set and dict; the others are immutable.

Use target or frequency encoding with cross-validation, hashing, embeddings or grouping rare categories, instead of huge one-hot matrices.

The square root of the variance; it measures how far values typically sit from the mean, in the same units as the data.

An iterable can produce an iterator (it has __iter__); an iterator has __next__ and remembers its position. A for-loop calls iter() on the iterable and then next() repeatedly.

Read in chunks or use usecols and dtypes, convert strings to category, avoid apply with row loops in favour of vectorised ops, and consider Polars, Dask or Parquet storage.

Bagging trains models in parallel on bootstrap samples and averages them (reduces variance, e.g. Random Forest). Boosting trains models one after another, each fixing the previous errors (reduces bias, e.g. XGBoost).

Package the model behind an API with versioning, track latency and errors, monitor input drift and prediction quality, and automate retraining with rollback.

map works element-wise on a Series, apply works on a Series or along a DataFrame axis, and applymap (now DataFrame.map) applies a function to every cell.

Calibration checks whether predicted probabilities match real frequencies (e.g. 70% predictions are right about 70% of the time). Fix with Platt scaling or isotonic regression.

It assigns points to the nearest centroid and updates centroids until stable. Choose k with the elbow method, silhouette score or business context.

Use grid search, random search or Bayesian optimisation with cross-validation, starting with a coarse random search and narrowing the promising ranges.

Correlation measures linear association between variables (-1 to 1). A third factor, reverse causality or chance can create it, so causation needs controlled experiments.

It computes the gradient of the loss with respect to every weight by applying the chain rule backwards through the network, so gradient descent can update them.

Term Frequency × Inverse Document Frequency: a word scores high if it is frequent in one document but rare across all documents. A classic way to turn text into numeric features.

Frequentists treat parameters as fixed and use long-run frequencies; Bayesians treat parameters as distributions updated with data via priors and posteriors, giving direct probability statements.

During training it randomly switches off a fraction of neurons so the network cannot depend on any single unit, acting as a regulariser against overfitting.

ReLU for hidden layers, sigmoid for binary outputs, softmax for multi-class outputs and tanh when zero-centred outputs help; Leaky ReLU avoids dead neurons.

Predictors that are highly correlated make coefficients unstable. Detect with VIF above about 5-10, then drop or combine features, use PCA or apply regularisation.

Transformers use self-attention to process all tokens in parallel and capture long-range context, making them faster to train and more scalable than sequential RNNs.

A table of true positives, false positives, true negatives and false negatives from which accuracy, precision, recall and F1 are calculated.

An epoch is one pass over the training data; a batch is the subset used for one update; an iteration is one weight update on a batch.

This is overfitting. Check for data leakage, simplify the model, add regularization, get more data or augment it, use cross-validation, early stopping and feature selection.

First understand why data is missing. Then drop rows or columns with too many nulls, impute with mean/median/mode or a model (KNN), add a 'was_missing' flag, and fit imputers on training data only.

It projects data onto orthogonal directions of maximum variance to reduce dimensions, remove correlation and speed up models, at some cost of interpretability.

It normalises layer inputs per mini-batch and then rescales them, stabilising and speeding up training and allowing higher learning rates.

A network that compresses inputs into a small latent code and reconstructs them; used for denoising, dimensionality reduction and anomaly detection.

Accuracy, F1 and confusion matrices for classification, BLEU or ROUGE for generation and summarisation, perplexity for language models, plus human review.

Use stratified splits, class weights, resampling (SMOTE or undersampling) and threshold tuning, and judge the model with PR-AUC or F1 instead of accuracy.

As the number of independent trials grows, the sample average converges to the true expected value, which is why larger samples give more stable estimates.

Pearson measures linear relationships on continuous data; Spearman uses ranks, so it captures monotonic relationships and is less sensitive to outliers.

Activations add non-linearity so networks can learn complex patterns. ReLU (max(0, x)) is cheap and reduces vanishing gradients compared with sigmoid/tanh; Leaky ReLU avoids 'dead' neurons.

Train fits the model, validation tunes hyperparameters and model choice, and the untouched test set gives a final unbiased estimate of performance.

A range built from sample data that would contain the true parameter in a stated percentage (for example 95%) of repeated samples. It is not the probability that the parameter lies inside this one interval.

RNNs process sequences with a hidden state but struggle with long dependencies. LSTMs add gates that control what to remember or forget, easing the vanishing gradient.

It fits a line minimising squared error. Assumes linearity, independent errors, constant variance (homoscedasticity), normally distributed residuals and no strong multicollinearity.

A method is a function that belongs to an object or class and receives the instance (self) as its first argument.

Discriminative models learn P(y given x) directly (logistic regression, SVM). Generative models learn the joint P(x, y) and can generate data (Naive Bayes, GANs).

Use filter methods (correlation, chi-square), wrapper methods (RFE), embedded methods (Lasso, tree importance) and domain knowledge, validating each choice with cross-validation.

Add a classification head, freeze or gradually unfreeze layers, use a small learning rate, train on labelled data with validation, and track F1 per class.

loc selects by label (row and column names, inclusive of the end label); iloc selects by integer position (end-exclusive).

Monitor input distributions and live metrics, compare to training baselines, and retrain or recalibrate on fresh data when performance drops.

List: ordered and mutable. Tuple: ordered and immutable (hashable, slightly faster). Set: unordered, unique items, fast membership checks. Dict: key–value pairs with fast lookup by key.

*args collects extra positional arguments into a tuple; **kwargs collects extra keyword arguments into a dict. They make function signatures flexible and are common in wrappers and decorators.

Regression predicts continuous values such as price; classification predicts discrete categories such as spam or not spam.

The model memorises noise and fails on new data. Prevent it with more data, simpler models, regularisation, cross-validation, early stopping and dropout.

They store same-type values in contiguous memory and run vectorised operations in compiled C, avoiding Python loop overhead and per-object type checks.

Type I = false positive (rejecting a true null), probability α. Type II = false negative (missing a real effect), probability β. Power = 1 − β.

The probability of seeing a result at least as extreme as the observed one if the null hypothesis were true. A small p-value is evidence against the null, not proof that the alternative is true.

A function using yield that produces values lazily, one at a time. It saves memory when processing large files or streams because the full sequence is never held in RAM.

L1 (Lasso) adds absolute weights and can shrink some to zero, giving feature selection. L2 (Ridge) adds squared weights and shrinks them smoothly without removing features.

Power is the probability of detecting a real effect. Increase it with a larger sample, a bigger effect size, lower noise or a higher significance level.

The distribution of sample means approaches a normal distribution as the sample size grows (roughly 30+), whatever the population's shape. It lets us build confidence intervals and run hypothesis tests on means.

Reusing a model pretrained on a large dataset and fine-tuning it on a smaller task-specific dataset, saving data, time and compute.

Putting features on comparable ranges using standardisation or min-max scaling. Distance-based and gradient-based models (KNN, SVM, regression, neural nets) need it; tree models do not.

A forest averages many trees trained on bootstrapped rows and random feature subsets, which cuts variance and overfitting at the cost of interpretability.

An optimisation method that updates weights in the opposite direction of the loss gradient. The learning rate sets the step size: too high and it diverges, too low and it is slow. Variants include SGD, mini-batch and Adam.

Supervised learning trains on labelled data to predict a target; unsupervised learning finds structure such as clusters or components in unlabelled data.

It predicts from the k nearest labelled points. It is simple but slow at prediction time, sensitive to scaling and irrelevant features, and weak in high dimensions.

Raising the threshold improves precision but lowers recall. Fraud detection leans to recall, while spam filters often prefer precision; choose by the cost of each error.

Training a meta-model on the predictions of several base models, often using cross-validated predictions to avoid leakage.

Define churn and the prediction window → build usage and billing features → handle imbalance → train gradient boosting → evaluate with PR-AUC and lift → explain with SHAP → deploy as batch scoring or an API → monitor drift → hand top-risk users to the retention team.

A symmetric bell-shaped distribution defined by mean and standard deviation, where about 68%, 95% and 99.7% of values lie within 1, 2 and 3 standard deviations.

CNNs use convolutions to find spatial patterns (images). RNNs/LSTMs process sequences step by step but struggle with long context. Transformers use self-attention to look at all tokens at once — parallel and better at long-range context; they power modern LLMs.

It tries candidate splits and picks the one that most reduces impurity — Gini or entropy (information gain) for classification, variance / MSE for regression.

Split the data into k folds, train on k−1 folds and validate on the remaining one, rotate and average the scores. It gives a more reliable estimate than one split and helps tune hyperparameters without touching the test set.

They build a list in one readable expression, for example [x*x for x in data if x > 0]. They are usually faster than an equivalent for-loop with append.

Bias occurs when the sample does not represent the population. Reduce it with random or stratified sampling, higher response rates and by checking sample demographics against the population.

It models the log-odds of a binary outcome as a linear function and passes it through a sigmoid to give probabilities, trained by maximising likelihood.

Define the hypothesis and one primary metric, calculate sample size from the baseline, minimum detectable effect, α and power, randomise users, run for full business cycles, check for sample-ratio mismatch, then test both statistical and practical significance.

Define churn and the prediction window, build behavioural features, split by time, train and compare models, evaluate with recall and lift, explain drivers and monitor drift after deployment.

Splitting text into units such as words, subwords or characters that are mapped to ids; subword methods like BPE handle rare words well.

Retrieval-Augmented Generation fetches relevant documents and gives them to an LLM as context, reducing hallucination and keeping answers grounded in current private data.

A set stores unique values with no keys; a dictionary stores key-value pairs with unique keys. Both use hashing, so lookups are O(1) on average.

The probability of seeing a result at least this extreme if the null hypothesis were true. A small p-value (for example < 0.05) is evidence against the null hypothesis.

Use the IQR rule (below Q1-1.5 IQR or above Q3+1.5 IQR), z-scores beyond 3, box plots or model-based methods such as Isolation Forest, then decide whether to cap, remove or keep them.