Deep Learning and NLP Interview Questions for Data Science
46 Deep Learning and NLP questions with answers for data science roles — 9 easy, 28 medium, 9 hard.
About Deep Learning and NLP
This topic covers neural network basics, training problems, CNNs, RNNs, transformers and practical NLP such as tokenisation, embeddings, classification and NER. Interviewers check that you understand why techniques work and how to debug training. It matters more for ML engineer, NLP and AI roles, but most data science interviews include a few questions.
Sheet progressStart practicing
3 QUESTIONS FREESign up to unlock the full question bank and track your progress.Unlock every question and answer, then continue practising company-wise and package-wise.
Word embeddings represent words as dense vectors, usually 100 to 300 numbers, so that words used in similar contexts get similar vectors. Word2Vec learns them by predicting a word from its neighbours or the neighbours from the word. Unlike one-hot or TF-IDF, embeddings capture meaning, so king and queen are close. Their limit is one vector per word, so bank has the same vector in river bank and bank loan.
04
What is the difference between stemming and lemmatisation?
Both reduce words to a base form. Stemming chops suffixes using rules, so studies becomes studi; it is fast but can produce non-words. Lemmatisation uses vocabulary and grammar to return the real dictionary form, so studies becomes study and better becomes good. Lemmatisation is more accurate but slower. With modern transformer models, neither is usually needed because subword tokenisers handle word forms.
Named entity recognition finds and labels spans of text such as person names, organisations, locations, dates and amounts. For example, in an invoice email it can pull out the vendor, invoice number and total. It is usually framed as token classification with BIO tags. spaCy and fine-tuned transformer models are common tools. Domain-specific entities, like medicine names, usually need custom labelled training data.
06
Write Keras code for a small binary classifier on tabular data.
Data augmentation creates new training examples by applying realistic changes to existing ones. For images this means flips, rotations, crops and brightness changes. For text it can mean synonym replacement, back-translation or random word dropping. For audio, adding noise or shifting pitch. It improves generalisation when data is limited. Changes must keep the label correct; flipping a digit 6 can turn it into a 9.
08
How would you quickly build a sentiment analysis model for product reviews?
Step 1: collect and label a sample of reviews, checking class balance. Step 2: build a baseline with TF-IDF and logistic regression to get a quick benchmark. Step 3: fine-tune a small pretrained transformer such as DistilBERT for better accuracy, or use a multilingual model if reviews mix languages. Step 4: evaluate with F1 per class and read wrong predictions. Step 5: handle sarcasm and neutral reviews explicitly.
09
Write Python code using Hugging Face transformers to run sentiment analysis on a list of reviews.
from transformers import pipeline
clf = pipeline(
'sentiment-analysis',
model='distilbert-base-uncased-finetuned-sst-2-english'
)
reviews = ['Delivery was fast and the product is great',
'Packaging was damaged and support did not reply']
for text, out in zip(reviews, clf(reviews, batch_size=16)):
print(out['label'], round(out['score'], 3), '-', text)
10
What is the difference between an epoch, a batch and an iteration?
Gradients shrink as they pass through many layers (for example with sigmoid), so early layers barely learn. ReLU, residual connections and good initialisation help.
13
Which activation functions do you know and when to use them?
ReLU for hidden layers, sigmoid for binary outputs, softmax for multi-class outputs and tanh when zero-centred outputs help; Leaky ReLU avoids dead neurons.
RNNs process sequences with a hidden state but struggle with long dependencies. LSTMs add gates that control what to remember or forget, easing the vanishing gradient.
Term Frequency × Inverse Document Frequency: a word scores high if it is frequent in one document but rare across all documents. A classic way to turn text into numeric features.
It computes the gradient of the loss with respect to every weight by applying the chain rule backwards through the network, so gradient descent can update them.
Convolution filters slide over the input to detect local patterns, pooling reduces size, and deeper layers combine edges into shapes and objects. Used mainly for images.
Accuracy, F1 and confusion matrices for classification, BLEU or ROUGE for generation and summarisation, perplexity for language models, plus human review.
It lets a model weigh which input positions matter most for each output, computing weighted combinations of values from query-key similarity. It is the core of transformers.
23
How do you choose a learning rate for a neural network?
During training it randomly switches off a fraction of neurons so the network cannot depend on any single unit, acting as a regulariser against overfitting.
25
Why is cross-entropy preferred over MSE for classification in neural networks?
Cross-entropy measures how far predicted probabilities are from the true class and strongly penalises confident wrong predictions. With a softmax or sigmoid output, it gives large, useful gradients when the model is wrong. MSE with sigmoid outputs gives very small gradients when predictions saturate, so learning slows down. Use binary cross-entropy for two classes, categorical cross-entropy for many classes, and MSE for regression targets.
26
What is the difference between SGD and Adam optimisers?
SGD updates weights using the gradient times a fixed learning rate, often with momentum. Adam keeps running averages of gradients and their squares, giving each parameter its own adaptive step size. Adam converges fast with little tuning and is the default for transformers. Well-tuned SGD with momentum sometimes generalises better on vision tasks. AdamW, which decouples weight decay, is the common choice for modern deep learning.
27
Tell me about a time you had to learn a new deep learning framework or tool quickly.
Situation: my internship project used PyTorch, but I had only used Keras. Task: build a text classifier within two weeks. Action: I followed the official tutorials for two days, rebuilt a small Keras model in PyTorch to compare, and asked a senior to review my training loop. Result: I delivered the model on time with 88 percent F1. I learned that rebuilding something familiar is the fastest way into a new tool.
28
Write a basic PyTorch training loop for one epoch.
model.train()
for X, y in train_loader:
X, y = X.to(device), y.to(device)
optimizer.zero_grad()
logits = model(X)
loss = loss_fn(logits, y)
loss.backward()
optimizer.step()
# evaluation
model.eval()
with torch.no_grad():
val_loss = sum(loss_fn(model(Xv.to(device)), yv.to(device)).item() for Xv, yv in val_loader)
29
Your neural network's training loss suddenly becomes NaN. What do you check?
Common causes are a learning rate that is too high, exploding gradients, bad input values, and unstable maths like log of zero. I would check the input data for NaN or infinite values, lower the learning rate, add gradient clipping, and make sure the loss uses the stable built-in version with logits. With mixed precision, I would check the loss scaler. Logging gradient norms per step helps find the exact point it breaks.
30
Your image classifier trained on 2,000 images memorises the training set within 5 epochs. What do you try?
With so little data, I would use transfer learning from a pretrained model like ResNet or EfficientNet and fine-tune only the top layers first. I would add data augmentation such as flips, crops and colour changes, use dropout and weight decay, and apply early stopping on validation loss. Collecting more labelled images or using a smaller model also helps. I would check for duplicate images across train and validation sets too.
31
What is subword tokenisation like BPE or WordPiece?
Subword tokenisers split text into frequent pieces between characters and full words. Byte Pair Encoding starts from characters and repeatedly merges the most frequent pairs. WordPiece, used by BERT, is similar but chooses merges by likelihood. Rare words like unhappiness become un, happi, ness, so there are no unknown words and the vocabulary stays around 30,000 to 100,000. Token counts also decide LLM cost and context use.
32
Training a transformer on a GPU fails with an out-of-memory error. What can you do?
Reduce the batch size and use gradient accumulation to keep the same effective batch. Shorten the maximum sequence length. Turn on mixed precision training with float16 or bfloat16. Use gradient checkpointing, which saves memory by recomputing activations. Freeze some layers or switch to a smaller model, or use parameter-efficient fine-tuning like LoRA. Also make sure no old tensors stay on the GPU between runs.
33
A deep network you built from scratch does not learn at all; the loss stays flat from the first epoch. Could weight initialisation be the reason?
If weights start too large, activations and gradients explode; if too small, they vanish, and training stalls. If all weights are equal, every neuron learns the same thing. Good schemes keep the variance of activations stable across layers: Xavier or Glorot initialisation for tanh and sigmoid, and He initialisation for ReLU. Frameworks use sensible defaults, but it matters when building custom layers or very deep networks.
34
How do you mentor junior data scientists on your team?
Situation: two freshers joined my team and struggled with messy real data. Task: help them deliver independently. Action: I paired with them on the first project, set up weekly code reviews, shared a checklist for validation and leakage, and gave them small end-to-end tasks with clear success metrics. Result: within three months both owned models in production. I focus on teaching how to debug and ask questions, not just giving answers.
35
What is the difference between layer normalisation and batch normalisation?
Batch normalisation normalises each feature using the mean and variance across the batch, so it depends on batch size and behaves differently in training and inference. Layer normalisation normalises across the features of each single example, so it works with any batch size and with variable-length sequences. That is why transformers use layer norm, often in the pre-norm position, while CNNs traditionally use batch norm.
36
You must classify support messages into 40 categories, but some categories have only 20 examples. What do you do?
Start with a pretrained sentence embedding model plus a simple classifier, or fine-tune a small transformer with class weights. For the rare categories, try few-shot approaches such as SetFit or an LLM with examples in the prompt, and generate or collect more examples. Merge categories that are rarely distinguishable. Report per-class F1 rather than overall accuracy, and route low-confidence predictions to a human queue.
37
Tell me about a time you had to explain a deep learning model's mistake to a client or stakeholder.
Situation: our document classifier mislabelled several legal notices, and the client was upset. Task: explain clearly and restore confidence. Action: I showed the misclassified examples, explained that they used a format absent from training data, and avoided jargon. I proposed adding those samples, a confidence threshold for human review and weekly error reports. Result: errors on that format fell sharply and the client extended the project.
38
How would you fine-tune a pretrained language model for classification?
Add a classification head, freeze or gradually unfreeze layers, use a small learning rate, train on labelled data with validation, and track F1 per class.
39
CNN vs RNN vs Transformer — what is the difference?
CNNs use convolutions to find spatial patterns (images). RNNs/LSTMs process sequences step by step but struggle with long context. Transformers use self-attention to look at all tokens at once — parallel and better at long-range context; they power modern LLMs.
Transformers use self-attention to process all tokens in parallel and capture long-range context, making them faster to train and more scalable than sequential RNNs.
BERT is an encoder-only transformer that reads text in both directions at once. It was pretrained with masked language modelling, where about 15 percent of tokens are hidden and the model predicts them, plus next sentence prediction in the original version. This gives contextual embeddings, so a word's vector depends on its sentence. You fine-tune BERT with a small task head for classification, NER or question answering.
42
What is positional encoding in transformers and why is it needed?
Self-attention treats input tokens as a set, so without extra information it cannot tell word order: dog bites man and man bites dog look the same. Positional encoding adds position information to each token embedding. The original transformer used fixed sine and cosine patterns; BERT learned position embeddings; many modern LLMs use rotary embeddings, which encode relative positions and extend better to longer contexts.
43
What is the difference between encoder-only, decoder-only and encoder-decoder transformers?
Encoder-only models like BERT see the full input in both directions and are best for understanding tasks such as classification, NER and embeddings. Decoder-only models like GPT predict the next token using only past tokens, so they suit text generation and chat. Encoder-decoder models like T5 encode the input and then generate output, which fits translation and summarisation. Most modern LLMs are decoder-only because they scale well.
44
Describe a project where you worked with very little labelled data.
Situation: we had to classify 50,000 support tickets into 12 categories with only 600 labelled. Task: reach usable accuracy in a month. Action: I used a pretrained sentence embedding model with logistic regression, then active learning to pick the most uncertain tickets for agents to label in short daily batches. Result: with 2,000 labels we reached 86 percent macro F1. I learned that smart labelling beats waiting for perfect data.
45
Your NLP classifier works well on English reviews but fails on Hinglish reviews. What do you do?
First measure the gap with a labelled Hinglish test set. Hinglish mixes scripts and spellings, so tokenisers built for English split it badly. I would try a multilingual or Indic model such as mBERT, XLM-R or IndicBERT, normalise common spellings, and add labelled Hinglish data, possibly with transliteration. Evaluate per language segment and monitor the language mix in production, because it changes over time.
Mixed precision training uses 16-bit floats, float16 or bfloat16, for most operations while keeping a 32-bit master copy of the weights. It roughly halves memory use and speeds up training on modern GPUs with tensor cores. With float16, a loss scaler multiplies the loss to avoid gradients underflowing to zero. bfloat16 has a wider range and usually needs no scaling. Final accuracy is normally the same as full precision.
Frequently asked questions
How important is Deep Learning and NLP in Data Science interviews?
This topic covers neural network basics, training problems, CNNs, RNNs, transformers and practical NLP such as tokenisation, embeddings, classification and NER. Interviewers check that you understand why techniques work and how to debug training. It matters more for ML engineer, NLP and AI roles, but most data science interviews include a few questions.
How should I use the Deep Learning and NLP sheet?
Go top to bottom — easy to hard. Answer before opening each answer, tick what you solved and save misses to Last-Minute Revision.