Skip to content

Deep Learning and NLP Interview Questions for Data Science

46 Deep Learning and NLP questions with answers for data science roles — 9 easy, 28 medium, 9 hard.

About Deep Learning and NLP

This topic covers neural network basics, training problems, CNNs, RNNs, transformers and practical NLP such as tokenisation, embeddings, classification and NER. Interviewers check that you understand why techniques work and how to debug training. It matters more for ML engineer, NLP and AI roles, but most data science interviews include a few questions.

Sheet progressStart practicing
3 QUESTIONS FREESign up to unlock the full question bank and track your progress.Unlock every question and answer, then continue practising company-wise and package-wise.
3 questions free Unlock the rest in seconds
01

What is tokenisation in NLP?

Save to ▾
Show answer
Splitting text into units such as words, subwords or characters that are mapped to ids; subword methods like BPE handle rare words well.
02

What is a neural network?

Save to ▾
Show answer
Layers of connected neurons that apply weights, biases and activation functions to learn complex patterns, trained by backpropagation.
03

What are word embeddings like Word2Vec?

Save to ▾
Show answer
Word embeddings represent words as dense vectors, usually 100 to 300 numbers, so that words used in similar contexts get similar vectors. Word2Vec learns them by predicting a word from its neighbours or the neighbours from the word. Unlike one-hot or TF-IDF, embeddings capture meaning, so king and queen are close. Their limit is one vector per word, so bank has the same vector in river bank and bank loan.
04

What is the difference between stemming and lemmatisation?

Save to ▾
Show answer
Both reduce words to a base form. Stemming chops suffixes using rules, so studies becomes studi; it is fast but can produce non-words. Lemmatisation uses vocabulary and grammar to return the real dictionary form, so studies becomes study and better becomes good. Lemmatisation is more accurate but slower. With modern transformer models, neither is usually needed because subword tokenisers handle word forms.
05

What is named entity recognition?

Save to ▾
Show answer
Named entity recognition finds and labels spans of text such as person names, organisations, locations, dates and amounts. For example, in an invoice email it can pull out the vendor, invoice number and total. It is usually framed as token classification with BIO tags. spaCy and fine-tuned transformer models are common tools. Domain-specific entities, like medicine names, usually need custom labelled training data.
06

Write Keras code for a small binary classifier on tabular data.

Save to ▾
Show answer
from tensorflow import keras model = keras.Sequential([ keras.layers.Input(shape=(X_train.shape[1],)), keras.layers.Dense(64, activation='relu'), keras.layers.Dropout(0.3), keras.layers.Dense(1, activation='sigmoid') ]) model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['AUC']) model.fit(X_train, y_train, validation_split=0.2, epochs=20, batch_size=64)
07

What is data augmentation?

Save to ▾
Show answer
Data augmentation creates new training examples by applying realistic changes to existing ones. For images this means flips, rotations, crops and brightness changes. For text it can mean synonym replacement, back-translation or random word dropping. For audio, adding noise or shifting pitch. It improves generalisation when data is limited. Changes must keep the label correct; flipping a digit 6 can turn it into a 9.
08

How would you quickly build a sentiment analysis model for product reviews?

Save to ▾
Show answer
Step 1: collect and label a sample of reviews, checking class balance. Step 2: build a baseline with TF-IDF and logistic regression to get a quick benchmark. Step 3: fine-tune a small pretrained transformer such as DistilBERT for better accuracy, or use a multilingual model if reviews mix languages. Step 4: evaluate with F1 per class and read wrong predictions. Step 5: handle sarcasm and neutral reviews explicitly.
09

Write Python code using Hugging Face transformers to run sentiment analysis on a list of reviews.

Save to ▾
Show answer
from transformers import pipeline clf = pipeline( 'sentiment-analysis', model='distilbert-base-uncased-finetuned-sst-2-english' ) reviews = ['Delivery was fast and the product is great', 'Packaging was damaged and support did not reply'] for text, out in zip(reviews, clf(reviews, batch_size=16)): print(out['label'], round(out['score'], 3), '-', text)
10

What is the difference between an epoch, a batch and an iteration?

Save to ▾
Show answer
An epoch is one pass over the training data; a batch is the subset used for one update; an iteration is one weight update on a batch.
11

Why do we use activation functions? Why is ReLU popular?

Save to ▾
Show answer
Activations add non-linearity so networks can learn complex patterns. ReLU (max(0, x)) is cheap and reduces vanishing gradients compared with sigmoid/tanh; Leaky ReLU avoids 'dead' neurons.
12

What is the vanishing gradient problem?

Save to ▾
Show answer
Gradients shrink as they pass through many layers (for example with sigmoid), so early layers barely learn. ReLU, residual connections and good initialisation help.
13

Which activation functions do you know and when to use them?

Save to ▾
Show answer
ReLU for hidden layers, sigmoid for binary outputs, softmax for multi-class outputs and tanh when zero-centred outputs help; Leaky ReLU avoids dead neurons.
14

What are RNNs and why did LSTMs improve them?

Save to ▾
Show answer
RNNs process sequences with a hidden state but struggle with long dependencies. LSTMs add gates that control what to remember or forget, easing the vanishing gradient.
15

What is an autoencoder and what is it used for?

Save to ▾
Show answer
A network that compresses inputs into a small latent code and reconstructs them; used for denoising, dimensionality reduction and anomaly detection.
16

What is TF-IDF?

Save to ▾
Show answer
Term Frequency × Inverse Document Frequency: a word scores high if it is frequent in one document but rare across all documents. A classic way to turn text into numeric features.
17

What is batch normalisation?

Save to ▾
Show answer
It normalises layer inputs per mini-batch and then rescales them, stabilising and speeding up training and allowing higher learning rates.
18

What is backpropagation?

Save to ▾
Show answer
It computes the gradient of the loss with respect to every weight by applying the chain rule backwards through the network, so gradient descent can update them.
19

What is transfer learning?

Save to ▾
Show answer
Reusing a model pretrained on a large dataset and fine-tuning it on a smaller task-specific dataset, saving data, time and compute.
20

How does a CNN work?

Save to ▾
Show answer
Convolution filters slide over the input to detect local patterns, pooling reduces size, and deeper layers combine edges into shapes and objects. Used mainly for images.
21

What are common ways to evaluate NLP models?

Save to ▾
Show answer
Accuracy, F1 and confusion matrices for classification, BLEU or ROUGE for generation and summarisation, perplexity for language models, plus human review.
22

What is the attention mechanism?

Save to ▾
Show answer
It lets a model weigh which input positions matter most for each output, computing weighted combinations of values from query-key similarity. It is the core of transformers.
23

How do you choose a learning rate for a neural network?

Save to ▾
Show answer
Start with a sensible default such as 1e-3 for Adam, run a learning-rate range test, and use schedulers or warm-up while watching the loss curve.
24

How does dropout work?

Save to ▾
Show answer
During training it randomly switches off a fraction of neurons so the network cannot depend on any single unit, acting as a regulariser against overfitting.
25

Why is cross-entropy preferred over MSE for classification in neural networks?

Save to ▾
Show answer
Cross-entropy measures how far predicted probabilities are from the true class and strongly penalises confident wrong predictions. With a softmax or sigmoid output, it gives large, useful gradients when the model is wrong. MSE with sigmoid outputs gives very small gradients when predictions saturate, so learning slows down. Use binary cross-entropy for two classes, categorical cross-entropy for many classes, and MSE for regression targets.
26

What is the difference between SGD and Adam optimisers?

Save to ▾
Show answer
SGD updates weights using the gradient times a fixed learning rate, often with momentum. Adam keeps running averages of gradients and their squares, giving each parameter its own adaptive step size. Adam converges fast with little tuning and is the default for transformers. Well-tuned SGD with momentum sometimes generalises better on vision tasks. AdamW, which decouples weight decay, is the common choice for modern deep learning.
27

Tell me about a time you had to learn a new deep learning framework or tool quickly.

Save to ▾
Show answer
Situation: my internship project used PyTorch, but I had only used Keras. Task: build a text classifier within two weeks. Action: I followed the official tutorials for two days, rebuilt a small Keras model in PyTorch to compare, and asked a senior to review my training loop. Result: I delivered the model on time with 88 percent F1. I learned that rebuilding something familiar is the fastest way into a new tool.
28

Write a basic PyTorch training loop for one epoch.

Save to ▾
Show answer
model.train() for X, y in train_loader: X, y = X.to(device), y.to(device) optimizer.zero_grad() logits = model(X) loss = loss_fn(logits, y) loss.backward() optimizer.step() # evaluation model.eval() with torch.no_grad(): val_loss = sum(loss_fn(model(Xv.to(device)), yv.to(device)).item() for Xv, yv in val_loader)
29

Your neural network's training loss suddenly becomes NaN. What do you check?

Save to ▾
Show answer
Common causes are a learning rate that is too high, exploding gradients, bad input values, and unstable maths like log of zero. I would check the input data for NaN or infinite values, lower the learning rate, add gradient clipping, and make sure the loss uses the stable built-in version with logits. With mixed precision, I would check the loss scaler. Logging gradient norms per step helps find the exact point it breaks.
30

Your image classifier trained on 2,000 images memorises the training set within 5 epochs. What do you try?

Save to ▾
Show answer
With so little data, I would use transfer learning from a pretrained model like ResNet or EfficientNet and fine-tune only the top layers first. I would add data augmentation such as flips, crops and colour changes, use dropout and weight decay, and apply early stopping on validation loss. Collecting more labelled images or using a smaller model also helps. I would check for duplicate images across train and validation sets too.
31

What is subword tokenisation like BPE or WordPiece?

Save to ▾
Show answer
Subword tokenisers split text into frequent pieces between characters and full words. Byte Pair Encoding starts from characters and repeatedly merges the most frequent pairs. WordPiece, used by BERT, is similar but chooses merges by likelihood. Rare words like unhappiness become un, happi, ness, so there are no unknown words and the vocabulary stays around 30,000 to 100,000. Token counts also decide LLM cost and context use.
32

Training a transformer on a GPU fails with an out-of-memory error. What can you do?

Save to ▾
Show answer
Reduce the batch size and use gradient accumulation to keep the same effective batch. Shorten the maximum sequence length. Turn on mixed precision training with float16 or bfloat16. Use gradient checkpointing, which saves memory by recomputing activations. Freeze some layers or switch to a smaller model, or use parameter-efficient fine-tuning like LoRA. Also make sure no old tensors stay on the GPU between runs.
33

A deep network you built from scratch does not learn at all; the loss stays flat from the first epoch. Could weight initialisation be the reason?

Save to ▾
Show answer
If weights start too large, activations and gradients explode; if too small, they vanish, and training stalls. If all weights are equal, every neuron learns the same thing. Good schemes keep the variance of activations stable across layers: Xavier or Glorot initialisation for tanh and sigmoid, and He initialisation for ReLU. Frameworks use sensible defaults, but it matters when building custom layers or very deep networks.
34

How do you mentor junior data scientists on your team?

Save to ▾
Show answer
Situation: two freshers joined my team and struggled with messy real data. Task: help them deliver independently. Action: I paired with them on the first project, set up weekly code reviews, shared a checklist for validation and leakage, and gave them small end-to-end tasks with clear success metrics. Result: within three months both owned models in production. I focus on teaching how to debug and ask questions, not just giving answers.
35

What is the difference between layer normalisation and batch normalisation?

Save to ▾
Show answer
Batch normalisation normalises each feature using the mean and variance across the batch, so it depends on batch size and behaves differently in training and inference. Layer normalisation normalises across the features of each single example, so it works with any batch size and with variable-length sequences. That is why transformers use layer norm, often in the pre-norm position, while CNNs traditionally use batch norm.
36

You must classify support messages into 40 categories, but some categories have only 20 examples. What do you do?

Save to ▾
Show answer
Start with a pretrained sentence embedding model plus a simple classifier, or fine-tune a small transformer with class weights. For the rare categories, try few-shot approaches such as SetFit or an LLM with examples in the prompt, and generate or collect more examples. Merge categories that are rarely distinguishable. Report per-class F1 rather than overall accuracy, and route low-confidence predictions to a human queue.
37

Tell me about a time you had to explain a deep learning model's mistake to a client or stakeholder.

Save to ▾
Show answer
Situation: our document classifier mislabelled several legal notices, and the client was upset. Task: explain clearly and restore confidence. Action: I showed the misclassified examples, explained that they used a format absent from training data, and avoided jargon. I proposed adding those samples, a confidence threshold for human review and weekly error reports. Result: errors on that format fell sharply and the client extended the project.
38

How would you fine-tune a pretrained language model for classification?

Save to ▾
Show answer
Add a classification head, freeze or gradually unfreeze layers, use a small learning rate, train on labelled data with validation, and track F1 per class.
39

CNN vs RNN vs Transformer — what is the difference?

Save to ▾
Show answer
CNNs use convolutions to find spatial patterns (images). RNNs/LSTMs process sequences step by step but struggle with long context. Transformers use self-attention to look at all tokens at once — parallel and better at long-range context; they power modern LLMs.
40

How do transformers differ from RNNs?

Save to ▾
Show answer
Transformers use self-attention to process all tokens in parallel and capture long-range context, making them faster to train and more scalable than sequential RNNs.
41

What is BERT and how is it pretrained?

Save to ▾
Show answer
BERT is an encoder-only transformer that reads text in both directions at once. It was pretrained with masked language modelling, where about 15 percent of tokens are hidden and the model predicts them, plus next sentence prediction in the original version. This gives contextual embeddings, so a word's vector depends on its sentence. You fine-tune BERT with a small task head for classification, NER or question answering.
42

What is positional encoding in transformers and why is it needed?

Save to ▾
Show answer
Self-attention treats input tokens as a set, so without extra information it cannot tell word order: dog bites man and man bites dog look the same. Positional encoding adds position information to each token embedding. The original transformer used fixed sine and cosine patterns; BERT learned position embeddings; many modern LLMs use rotary embeddings, which encode relative positions and extend better to longer contexts.
43

What is the difference between encoder-only, decoder-only and encoder-decoder transformers?

Save to ▾
Show answer
Encoder-only models like BERT see the full input in both directions and are best for understanding tasks such as classification, NER and embeddings. Decoder-only models like GPT predict the next token using only past tokens, so they suit text generation and chat. Encoder-decoder models like T5 encode the input and then generate output, which fits translation and summarisation. Most modern LLMs are decoder-only because they scale well.
44

Describe a project where you worked with very little labelled data.

Save to ▾
Show answer
Situation: we had to classify 50,000 support tickets into 12 categories with only 600 labelled. Task: reach usable accuracy in a month. Action: I used a pretrained sentence embedding model with logistic regression, then active learning to pick the most uncertain tickets for agents to label in short daily batches. Result: with 2,000 labels we reached 86 percent macro F1. I learned that smart labelling beats waiting for perfect data.
45

Your NLP classifier works well on English reviews but fails on Hinglish reviews. What do you do?

Save to ▾
Show answer
First measure the gap with a labelled Hinglish test set. Hinglish mixes scripts and spellings, so tokenisers built for English split it badly. I would try a multilingual or Indic model such as mBERT, XLM-R or IndicBERT, normalise common spellings, and add labelled Hinglish data, possibly with transliteration. Evaluate per language segment and monitor the language mix in production, because it changes over time.
46

What is mixed precision training?

Save to ▾
Show answer
Mixed precision training uses 16-bit floats, float16 or bfloat16, for most operations while keeping a 32-bit master copy of the weights. It roughly halves memory use and speeds up training on modern GPUs with tensor cores. With float16, a loss scaler multiplies the loss to avoid gradients underflowing to zero. bfloat16 has a wider range and usually needs no scaling. Final accuracy is normally the same as full precision.

Frequently asked questions

How important is Deep Learning and NLP in Data Science interviews?

This topic covers neural network basics, training problems, CNNs, RNNs, transformers and practical NLP such as tokenisation, embeddings, classification and NER. Interviewers check that you understand why techniques work and how to debug training. It matters more for ML engineer, NLP and AI roles, but most data science interviews include a few questions.

How should I use the Deep Learning and NLP sheet?

Go top to bottom — easy to hard. Answer before opening each answer, tick what you solved and save misses to Last-Minute Revision.