HSBC Data Science Interview Questions & Answers

It models the log-odds of a binary outcome as a linear function and passes it through a sigmoid to give probabilities, trained by maximising likelihood.

Read in chunks or use usecols and dtypes, convert strings to category, avoid apply with row loops in favour of vectorised ops, and consider Polars, Dask or Parquet storage.

L1 (Lasso) adds absolute weights and can shrink some to zero, giving feature selection. L2 (Ridge) adds squared weights and shrinks them smoothly without removing features.

They store same-type values in contiguous memory and run vectorised operations in compiled C, avoiding Python loop overhead and per-object type checks.

Convolution filters slide over the input to detect local patterns, pooling reduces size, and deeper layers combine edges into shapes and objects. Used mainly for images.

Layers of connected neurons that apply weights, biases and activation functions to learn complex patterns, trained by backpropagation.

The model memorises noise and fails on new data. Prevent it with more data, simpler models, regularisation, cross-validation, early stopping and dropout.

A decorator wraps a function to add behaviour such as logging, timing or caching without changing its code. It is applied with the @name syntax above the function.

It assigns points to the nearest centroid and updates centroids until stable. Choose k with the elbow method, silhouette score or business context.

A range built from sample data that would contain the true parameter in a stated percentage (for example 95%) of repeated samples. It is not the probability that the parameter lies inside this one interval.

Bagging trains models in parallel on bootstrap samples and averages them (reduces variance, e.g. Random Forest). Boosting trains models one after another, each fixing the previous errors (reduces bias, e.g. XGBoost).

Transformers use self-attention to process all tokens in parallel and capture long-range context, making them faster to train and more scalable than sequential RNNs.

Train fits the model, validation tunes hyperparameters and model choice, and the untouched test set gives a final unbiased estimate of performance.

Parametric tests assume a distribution (usually normal) and use means; non-parametric tests such as Mann-Whitney use ranks and make fewer assumptions.

Small anonymous one-line functions, handy as arguments to sort, map, filter or Pandas apply.

It projects data onto orthogonal directions of maximum variance to reduce dimensions, remove correlation and speed up models, at some cost of interpretability.

It normalises layer inputs per mini-batch and then rescales them, stabilising and speeding up training and allowing higher learning rates.

Accuracy, F1 and confusion matrices for classification, BLEU or ROUGE for generation and summarisation, perplexity for language models, plus human review.

A mutex in CPython that lets only one thread execute Python bytecode at a time. Use multiprocessing for CPU-bound work and threads or asyncio for I/O-bound work.

pd.pivot_table(df, index='region', columns='month', values='sales', aggfunc='sum') reshapes long data into a summary matrix; melt() does the reverse.

map works element-wise on a Series, apply works on a Series or along a DataFrame axis, and applymap (now DataFrame.map) applies a function to every cell.

Type I = false positive (rejecting a true null), probability α. Type II = false negative (missing a real effect), probability β. Power = 1 − β.

Use the IQR rule (below Q1-1.5 IQR or above Q3+1.5 IQR), z-scores beyond 3, box plots or model-based methods such as Isolation Forest, then decide whether to cap, remove or keep them.

A measure of asymmetry. Right-skewed data has a long tail on the right (mean greater than median); left-skewed is the opposite. Log transforms often reduce right skew.

Use target or frequency encoding with cross-validation, hashing, embeddings or grouping rare categories, instead of huge one-hot matrices.

During training it randomly switches off a fraction of neurons so the network cannot depend on any single unit, acting as a regulariser against overfitting.

Define churn and the prediction window → build usage and billing features → handle imbalance → train gradient boosting → evaluate with PR-AUC and lift → explain with SHAP → deploy as batch scoring or an API → monitor drift → hand top-risk users to the retention team.

P(A given B) = P(B given A) x P(A) / P(B). In a medical test it converts test accuracy and disease prevalence into the real chance a positive result means disease.

Measure first with cProfile or timeit, then replace loops with NumPy or Pandas vectorised calls, use built-ins, caching and better data structures.

List: ordered and mutable. Tuple: ordered and immutable (hashable, slightly faster). Set: unordered, unique items, fast membership checks. Dict: key–value pairs with fast lookup by key.

It lets a model weigh which input positions matter most for each output, computing weighted combinations of values from query-key similarity. It is the core of transformers.

Predictors that are highly correlated make coefficients unstable. Detect with VIF above about 5-10, then drop or combine features, use PCA or apply regularisation.

Regression predicts continuous values such as price; classification predicts discrete categories such as spam or not spam.

It tries candidate splits and picks the one that most reduces impurity — Gini or entropy (information gain) for classification, variance / MSE for regression.

A function using yield that produces values lazily, one at a time. It saves memory when processing large files or streams because the full sequence is never held in RAM.

The ROC curve plots true positive rate against false positive rate across thresholds. AUC is the probability that a random positive is ranked above a random negative; 0.5 is chance.

The probability of seeing a result at least as extreme as the observed one if the null hypothesis were true. A small p-value is evidence against the null, not proof that the alternative is true.

It applies Bayes' theorem assuming features are independent given the class. The assumption is rarely true but the model is fast and works well for text.

Pearson measures linear relationships on continuous data; Spearman uses ranks, so it captures monotonic relationships and is less sensitive to outliers.

Correlation measures linear association between variables (-1 to 1). A third factor, reverse causality or chance can create it, so causation needs controlled experiments.

Start with a sensible default such as 1e-3 for Adam, run a learning-rate range test, and use schedulers or warm-up while watching the loss curve.

Use filter methods (correlation, chi-square), wrapper methods (RFE), embedded methods (Lasso, tree importance) and domain knowledge, validating each choice with cross-validation.

Training a meta-model on the predictions of several base models, often using cross-validated predictions to avoid leakage.

Calibration checks whether predicted probabilities match real frequencies (e.g. 70% predictions are right about 70% of the time). Fix with Platt scaling or isotonic regression.

The rule that lets arrays of different shapes operate together by virtually stretching the smaller one, for example adding a scalar or a (3,) vector to a (4,3) matrix.

CNNs use convolutions to find spatial patterns (images). RNNs/LSTMs process sequences step by step but struggle with long context. Transformers use self-attention to look at all tokens at once — parallel and better at long-range context; they power modern LLMs.

*args collects extra positional arguments into a tuple; **kwargs collects extra keyword arguments into a dict. They make function signatures flexible and are common in wrappers and decorators.

Retrieval-Augmented Generation fetches relevant documents and gives them to an LLM as context, reducing hallucination and keeping answers grounded in current private data.

Bias is error from overly simple assumptions (underfitting). Variance is sensitivity to the training data (overfitting). More model complexity lowers bias but raises variance — aim for the lowest total error using cross-validation and regularization.

First understand why data is missing. Then drop rows or columns with too many nulls, impute with mean/median/mode or a model (KNN), add a 'was_missing' flag, and fit imputers on training data only.

It checks whether observed category counts differ from expected counts, used for independence between two categorical variables and for goodness of fit.

Mean is the average, median is the middle value after sorting and mode is the most frequent value. Median is more robust to outliers.

Define churn and the prediction window, build behavioural features, split by time, train and compare models, evaluate with recall and lift, explain drivers and monitor drift after deployment.

Regularization adds a penalty on weights to reduce overfitting. L1 (Lasso) adds |w| and can push features to exactly zero. L2 (Ridge) adds w² and shrinks weights smoothly.

Use stratified splits, class weights, resampling (SMOTE or undersampling) and threshold tuning, and judge the model with PR-AUC or F1 instead of accuracy.

Supervised learning trains on labelled data to predict a target; unsupervised learning finds structure such as clusters or components in unlabelled data.

Tie metrics to business outcomes such as cost saved, show feature importance or SHAP values, give example predictions and be transparent about limitations.

Define the hypothesis and one primary metric, calculate sample size from the baseline, minimum detectable effect, α and power, randomise users, run for full business cycles, check for sample-ratio mismatch, then test both statistical and practical significance.

Monitor input distributions and live metrics, compare to training baselines, and retrain or recalibrate on fresh data when performance drops.

A table of true positives, false positives, true negatives and false negatives from which accuracy, precision, recall and F1 are calculated.

Gradients shrink as they pass through many layers (for example with sigmoid), so early layers barely learn. ReLU, residual connections and good initialisation help.

Splitting text into units such as words, subwords or characters that are mapped to ids; subword methods like BPE handle rare words well.

It predicts from the k nearest labelled points. It is simple but slow at prediction time, sensitive to scaling and irrelevant features, and weak in high dimensions.

SVM finds the maximum-margin separating hyperplane. Kernels such as RBF compute similarity in a higher-dimensional space implicitly, so non-linear boundaries are possible.

Discriminative models learn P(y given x) directly (logistic regression, SVM). Generative models learn the joint P(x, y) and can generate data (Naive Bayes, GANs).

== compares values for equality, while is checks whether two names point to the same object in memory. Use is mainly for None checks.

loc selects by label (row and column names, inclusive of the end label); iloc selects by integer position (end-exclusive).

A method is a function that belongs to an object or class and receives the instance (self) as its first argument.

Activations add non-linearity so networks can learn complex patterns. ReLU (max(0, x)) is cheap and reduces vanishing gradients compared with sigmoid/tanh; Leaky ReLU avoids 'dead' neurons.

Split the data into k folds, train on k−1 folds and validate on the remaining one, rotate and average the scores. It gives a more reliable estimate than one split and helps tune hyperparameters without touching the test set.