Data Science Interview Questions for IT services Companies

Show answer

Term Frequency × Inverse Document Frequency: a word scores high if it is frequent in one document but rare across all documents. A classic way to turn text into numeric features.

L1 vs L2 regularisation?

Show answer

L1 (Lasso) adds absolute weights and can shrink some to zero, giving feature selection. L2 (Ridge) adds squared weights and shrinks them smoothly without removing features.

What does CI/CD mean for machine learning projects?

MLOps and Model DeploymentMedium

Show answer

Continuous integration runs automatic checks on every code change: unit tests for feature code, data validation, linting and a quick training run on sample data. Continuous delivery packages the model and service, for example as a Docker image, and deploys it through staging to production after evaluation gates, such as the new model beating the current one on a holdout set. Some teams add continuous training, which retrains on new data automatically.

Write OpenCV code to read an image, convert it to grayscale, resize it and detect edges.

Computer VisionEasy

Show answer

import cv2 img = cv2.imread('part.jpg') # loaded as BGR if img is None: raise FileNotFoundError('part.jpg') gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) small = cv2.resize(gray, (224, 224), interpolation=cv2.INTER_AREA) blur = cv2.GaussianBlur(small, (5, 5), 0) edges = cv2.Canny(blur, 50, 150) cv2.imwrite('edges.png', edges)

What is the bias–variance trade-off?

Show answer

Bias is error from overly simple assumptions (underfitting). Variance is sensitivity to the training data (overfitting). More model complexity lowers bias but raises variance — aim for the lowest total error using cross-validation and regularization.

What is the difference between list, tuple, set and dict in Python?

Show answer

List: ordered and mutable. Tuple: ordered and immutable (hashable, slightly faster). Set: unordered, unique items, fast membership checks. Dict: key–value pairs with fast lookup by key.

Your sales forecast scores well in k-fold cross-validation but fails in production. What could be wrong with the validation?

Show answer

Random k-fold mixes future and past rows, so the model trains on data from after the period it predicts. That leaks information and inflates scores. Use time-based splits instead: train on an expanding or rolling window and validate on the next period, like TimeSeriesSplit in scikit-learn. Leave a gap between train and validation when features use lagged values, so the test reflects real forecasting conditions.

Your logging system was down for four days, leaving gaps in a daily series. How do you handle them before modelling?

Time Series & ForecastingEasy

Show answer

First make the gaps explicit by reindexing to a full daily calendar so missing days appear as NaN instead of silently disappearing. For short gaps, linear or time-based interpolation is usually fine. If there is strong weekly seasonality, fill using the same weekday from neighbouring weeks or a seasonal decomposition. Add a flag column marking imputed days, and never treat missing as zero unless zero really happened.

How do you pull large data from a SQL database into Pandas efficiently?

SQL for Data ScienceMedium

Show answer

Push as much work as possible into SQL: filter, aggregate and select only needed columns before pulling. Use pd.read_sql with a SQLAlchemy engine and chunksize to stream large results. For very large extracts, export to Parquet from the warehouse and read that, which is much faster than row-by-row transfer. Set dtypes like category early, and never use SELECT star on a big table from a notebook.

How do you profile and speed up slow Python code?

Python for Data ScienceMedium

Show answer

Measure first with cProfile or timeit, then replace loops with NumPy or Pandas vectorised calls, use built-ins, caching and better data structures.

What is the F1 score and when can it be misleading?

Machine LearningEasy

Show answer

F1 is the harmonic mean of precision and recall, so it is high only when both are high. It is useful for imbalanced classes where accuracy hides poor performance. It can mislead because it ignores true negatives and treats precision and recall as equally important. If missing a fraud case costs far more than a false alarm, use an F-beta score with beta above 1, or choose metrics based on real costs.

Correlation vs covariance?

Statistics and ProbabilityEasy

Show answer

Covariance shows the direction of a relationship but depends on units. Correlation is covariance scaled to −1…+1, so it shows direction and strength and is unit-free.

What is mean Average Precision (mAP) in object detection?

Show answer

For each class, detections are ranked by confidence and a precision-recall curve is computed, where a detection is a true positive if its IoU with an unmatched ground truth box passes a threshold. Average Precision is the area under that curve. mAP is the mean across classes. mAP@0.5 uses IoU 0.5, while mAP@0.5:0.95 averages over thresholds from 0.5 to 0.95 and rewards tighter boxes.

How do you perform feature selection?

Feature EngineeringMedium

Show answer

Use filter methods (correlation, chi-square), wrapper methods (RFE), embedded methods (Lasso, tree importance) and domain knowledge, validating each choice with cross-validation.

What is the difference between zero-shot and few-shot prompting?

Generative AI and LLMsEasy

Show answer

In zero-shot prompting you give only instructions, like classify this review as positive or negative. In few-shot prompting you also include a few worked examples of input and correct output, which shows the model the format and decision boundary. Few-shot usually improves consistency on custom labels, at the cost of more tokens. Pick examples that cover tricky cases and keep their format identical to what you want back.

What is the difference between a population and a sample?

Statistics and ProbabilityEasy

Show answer

A population is the full group you want to learn about, such as all customers of an app. A sample is the subset you actually observe, such as 5,000 surveyed customers. Numbers from the population are parameters, and numbers from a sample are statistics that estimate them. Good sampling, ideally random, lets you generalise from the sample to the population and measure the uncertainty using standard errors and confidence intervals.

Write Keras code for a small binary classifier on tabular data.

Deep Learning and NLPEasy

Show answer

from tensorflow import keras model = keras.Sequential([ keras.layers.Input(shape=(X_train.shape[1],)), keras.layers.Dense(64, activation='relu'), keras.layers.Dropout(0.3), keras.layers.Dense(1, activation='sigmoid') ]) model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['AUC']) model.fit(X_train, y_train, validation_split=0.2, epochs=20, batch_size=64)

What are dataclasses in Python and why are they useful in data projects?

Show answer

A dataclass, created with the @dataclass decorator, automatically generates __init__, __repr__ and __eq__ from annotated fields. It is a clean way to hold structured records such as a training config with learning rate, batch size and paths. Options like frozen=True make instances immutable, and default values reduce boilerplate. Compared with plain dictionaries, dataclasses give attribute access, type hints and clearer errors when a field is misspelled.

After one-hot encoding you have 2,000 features and training is slow. What do you do?

Feature EngineeringMedium

Show answer

Find which columns caused the explosion, usually a few high-cardinality categoricals. Group rare categories into Other, or switch those columns to frequency, target or hashing encoding. Use sparse matrices so zeros take no memory. Tree libraries like LightGBM and CatBoost can handle categoricals natively without one-hot. Then remove near-constant columns and check that validation scores hold up with the smaller feature set.

What are type hints in Python and why do data teams use them?

Show answer

Type hints annotate expected types, for example def clean(df: pd.DataFrame, cols: list[str]) -> pd.DataFrame. Python does not enforce them at runtime, but tools like mypy and IDEs use them to catch mistakes early and give better autocomplete. In data teams they make shared pipeline code easier to read and review. They matter most in production code and libraries, less in quick exploratory notebooks.

What are RNNs and why did LSTMs improve them?

Show answer

RNNs process sequences with a hidden state but struggle with long dependencies. LSTMs add gates that control what to remember or forget, easing the vanishing gradient.

What is batch normalisation?

Show answer

It normalises layer inputs per mini-batch and then rescales them, stabilising and speeding up training and allowing higher learning rates.

Write Python code to fit a Holt-Winters model and forecast the next 12 months.

Time Series & ForecastingMedium

Show answer

from statsmodels.tsa.holtwinters import ExponentialSmoothing s = df.set_index('month')['sales'].asfreq('MS') train, test = s[:-12], s[-12:] model = ExponentialSmoothing( train, trend='add', seasonal='mul', seasonal_periods=12 ).fit() forecast = model.forecast(12) mae = (forecast - test).abs().mean() print(forecast.round(1), 'MAE:', round(mae, 1))

What is stratified sampling in a train-test split?

Machine LearningEasy

Show answer

Stratified splitting keeps the class proportions the same in the training and test sets. If 5 percent of customers churn, both sets will have about 5 percent churners. Without it, a small or imbalanced dataset can give a test set with very few positives and unstable metrics. In scikit-learn you pass stratify=y to train_test_split or use StratifiedKFold for cross-validation.

Annotators labelling your images often disagree with each other. How do you handle this?

Computer VisionEasy

Show answer

First measure agreement, for example with Cohen's kappa or IoU between annotators' boxes, to find which classes are confusing. Usually the guidelines are unclear, so write precise rules with example images for edge cases and retrain annotators. Have multiple people label a subset and resolve conflicts by majority vote or an expert reviewer. Track ambiguous images separately, since even a perfect model cannot beat human label noise.

Why do residual connections make very deep networks easier to train?

Show answer

A residual block adds its input to its output, so the layers learn a residual F(x) and the block outputs F(x) plus x. If extra layers are not useful, they can learn to output near zero and the block behaves like identity. The skip path also lets gradients flow directly to early layers, reducing vanishing gradients. This is why ResNets with 50 or 100-plus layers train well where plain deep networks degrade.

When would you use MAE, MSE or RMSE for a regression model?

Machine LearningEasy

Show answer

MAE is the average absolute error; it is easy to explain and less sensitive to outliers. MSE squares errors, so large errors are punished heavily, and it is smooth for optimisation. RMSE is the square root of MSE, so it is in the same units as the target while still punishing big misses. Use RMSE when large errors are very costly, like stock-out forecasts, and MAE when outliers should not dominate.

What is recursive feature elimination?

Feature EngineeringMedium

Show answer

Recursive feature elimination trains a model, ranks features by importance or coefficient size, removes the weakest ones, and repeats until the desired number remains. RFECV picks the number of features using cross-validation. It considers features together rather than one at a time. The downsides are high compute cost on large data and unstable results when features are correlated, so check stability across folds.

How do one-stage and two-stage object detectors differ?

Show answer

Two-stage detectors, like Faster R-CNN, first propose candidate regions and then classify and refine each one, which gives strong accuracy, especially for small objects, but is slower. One-stage detectors, like YOLO and SSD, predict boxes and classes directly over a dense grid in a single pass, making them fast enough for real-time video. Modern one-stage models have closed most of the accuracy gap, so they dominate edge and video use cases.

What is the binomial distribution and when do you use it?

Statistics and ProbabilityEasy

Show answer

The binomial distribution gives the probability of getting k successes in n independent trials, each with the same success probability p. Its mean is n times p and its variance is n times p times (1 minus p). You use it for counts of yes-or-no outcomes, such as how many of 1,000 visitors convert or how many of 50 parts are defective. For large n it is well approximated by a normal distribution.

What is the receptive field of a neuron in a CNN?

Show answer

The receptive field is the region of the input image that can influence one activation. Early layers see small patches like edges, and deeper layers see larger regions, enabling recognition of whole objects. It grows with kernel size, depth, stride and pooling, and dilated convolutions increase it without extra parameters. If the receptive field is smaller than the objects you care about, the network cannot capture their full context.

What are Python's main built-in data types?

Show answer

int, float, complex, bool, str, list, tuple, set, dict and NoneType. Mutable ones are list, set and dict; the others are immutable.

What is a model registry and why do teams use one?

MLOps and Model DeploymentEasy

Show answer

A model registry is a central store for trained model versions along with their metadata: training data version, code commit, parameters, metrics and owner. Models move through stages such as staging, production and archived, with approvals. It gives one source of truth for which model is live, makes rollback to a previous version easy and supports audits. Tools like MLflow Model Registry are commonly used.

What is the difference between a Pandas Series and a DataFrame?

Show answer

A Series is a one-dimensional labelled array: one column of values with an index. A DataFrame is a two-dimensional table made of several Series that share the same index, and each column can have its own dtype. Selecting one column, like df['price'], gives you a Series. Selecting with a list, like df[['price']], gives you a DataFrame. Many methods work on both, but the output shape is different.

What is a neural network?

Deep Learning and NLPEasy

Show answer

Layers of connected neurons that apply weights, biases and activation functions to learn complex patterns, trained by backpropagation.

How do you evaluate clustering when you have no labels?

Show answer

Use internal metrics: the silhouette score compares how close a point is to its own cluster versus the nearest other cluster, and the Davies-Bouldin index measures cluster overlap. The elbow method looks at within-cluster variance. But the real test is usefulness: are clusters stable across random seeds and samples, can the business describe them, and do they behave differently on outcomes like spend or churn?

A product manager wants to test five different button designs at the same time. How do you set it up?

A/B Testing & ExperimentationEasy

Show answer

Run an A/B/n test with control plus five variants and equal random allocation. Calculate sample size for the smallest effect worth detecting, remembering that each variant gets only a sixth of traffic, so the test takes longer. Plan the analysis upfront: compare each variant to control and adjust for multiple comparisons. If the team has strong priors, test fewer designs first to save time.

What are the control group and treatment group in an A/B test?

A/B Testing & ExperimentationEasy

Show answer

The control group sees the current experience, and the treatment group sees the change being tested, such as a new checkout button. Users are randomly assigned, so the only systematic difference between groups is the change itself. Any difference in metrics beyond random noise can then be attributed to the treatment. Without a proper control, you cannot separate the effect of your change from seasonality, marketing or other launches.

What is subword tokenisation like BPE or WordPiece?