Interview Insights · Pandas
The Pandas Question That Ninety Percent of Candidates Fumble
Quick summary — Pandas transform question
The fumble is not about syntax; it is about the difference between one value per group and one value per row. Most candidates know groupby and sum. The share question asks for something slightly different: a group-level number placed back on every original row, so you can divide each amount by it. Aggregating returns a smaller table with a different index, and assigning it back produces NaN or an error. groupby with transform keeps the original shape and solves the problem in one line. The follow-up questions then test the same idea with ranking, running totals and gaps between orders.
In this guide you will learn:
- The exact question and the three typical wrong attempts
- Why the groupby result does not line up with your original rows
- How transform broadcasts a group value back to every row
- Four follow-ups that use the same pattern: rank, cumsum, diff, top-N
- How to explain the shape problem so the interviewer hears understanding
SECTION 01The question, and the three ways candidates fumble it
Here is the question as I usually ask it. You have an orders table with customer_id, order_id and amount. Add a column called share that shows what fraction of that customer's total spend each order represents. The values for one customer should add up to 1.
It looks like a two-minute task. In practice, a large share of the candidates I ask get stuck, even ones who just answered a regular groupby question without trouble. The attempts fall into three patterns.
| Attempt | Code they write | What happens | Root cause |
|---|---|---|---|
| Assign the aggregate | df['total'] = df.groupby('customer_id')['amount'].sum() | Column is mostly NaN | Result is indexed by customer_id, not by row |
| Divide directly | df['amount'] / df.groupby('customer_id')['amount'].sum() | Misaligned or NaN output | Pandas aligns on index labels, which do not match |
| Loop over customers | for c in df['customer_id'].unique(): ... | Works, but slow and long | Avoids the shape problem instead of solving it |
| Merge back | Aggregate, rename, merge on customer_id | Correct but several steps | Valid, but misses the one-line tool |
import pandas as pd
df = pd.DataFrame({
'order_id': [101, 102, 103, 104, 105],
'customer_id': ['C1', 'C1', 'C2', 'C2', 'C2'],
'amount': [200, 800, 100, 100, 300]
})
totals = df.groupby('customer_id')['amount'].sum()
print(totals)
# customer_id
# C1 1000
# C2 500
df['total'] = totals
print(df['total'])
# 0 NaN
# 1 NaN ... every row is NaN
# Index of df is 0..4, index of totals is C1, C2.
# No labels match, so nothing is assigned.import pandas as pd
totals = (df.groupby('customer_id', as_index=False)['amount']
.sum()
.rename(columns={'amount': 'customer_total'}))
out = df.merge(totals, on='customer_id', how='left')
out['share'] = out['amount'] / out['customer_total']
print(out)
# order_id customer_id amount customer_total share
# 0 101 C1 200 1000 0.2
# 1 102 C1 800 1000 0.8
# 2 103 C2 100 500 0.2
# 3 104 C2 100 500 0.2
# 4 105 C2 300 500 0.6
# Correct. But four steps for a one-line idea.
The merge-back version is a perfectly acceptable answer, and I would not reject anyone for it. But when I follow up with 'can you do it without a merge?', candidates who do not know transform usually stop there.
SECTION 02The transform answer, and how to explain it
groupby followed by transform runs a function on each group, just like an aggregation, but it returns a result with the same index and length as the original DataFrame. Every row receives the value computed for its own group. That is exactly what the share question needs.
When you give this answer, explain the shape. Saying 'sum would give me one row per customer, but transform gives me the customer total on every order row' tells the interviewer you understand why it works, not just that it works.
| Method | Output length | Output index | Typical use |
|---|---|---|---|
| groupby().agg / sum | One row per group | Group keys | Summary tables and reports |
| groupby().transform | Same as original | Original index | Group values placed on each row |
| groupby().apply | Depends on the function | Varies, can be confusing | Complex logic with no simpler option |
| groupby().filter | Rows from groups that pass | Original index | Keep groups meeting a condition |
import pandas as pd
df['share'] = df['amount'] / df.groupby('customer_id')['amount'].transform('sum')
print(df)
# order_id customer_id amount share
# 0 101 C1 200 0.2
# 1 102 C1 800 0.8
# 2 103 C2 100 0.2
# 3 104 C2 100 0.2
# 4 105 C2 300 0.6
# Sanity check: shares per customer add to 1
print(df.groupby('customer_id')['share'].sum())
# C1 1.0
# C2 1.0import pandas as pd
g = df.groupby('customer_id')['amount']
# Difference from the customer's average order
df['vs_avg'] = df['amount'] - g.transform('mean')
# Number of orders the customer has, on every row
df['n_orders'] = g.transform('size')
# Flag the customer's largest order
df['is_max'] = df['amount'] == g.transform('max')
# Keep only customers with 3+ orders (filter)
loyal = df.groupby('customer_id').filter(lambda x: len(x) >= 3)
print(loyal['customer_id'].unique()) # ['C2']
The sanity check at the end of the first tab is worth saying out loud in the interview. Checking that shares add up to 1 per customer shows you verify results instead of trusting the first output.
If you know SQL window functions, you can mention the connection: transform('sum') is the Pandas equivalent of SUM(amount) OVER (PARTITION BY customer_id). Interviewers who also work in SQL appreciate that link.
SECTION 03The follow-ups that test the same idea
Once a candidate solves the share question, I rarely stop. The follow-ups keep the same shape idea, a calculation within each group that returns one value per row, but use different tools. Candidates who truly understand transform pick these up quickly. Candidates who memorised the one-liner often stall again.
These four follow-ups come up often in analyst and data science rounds. Practise them on the same small DataFrame so you can see how each one behaves.
| Follow-up question | Pandas tool | SQL equivalent | Common slip |
|---|---|---|---|
| Rank each order within its customer by amount | groupby().rank(ascending=False) | RANK() OVER (PARTITION BY ...) | Forgetting how ties are ranked |
| Running total of spend per customer by date | sort_values then groupby().cumsum() | SUM() OVER (... ORDER BY date) | Not sorting by date first |
| Days since the customer's previous order | groupby()['date'].diff().dt.days | date - LAG(date) OVER (...) | Dates still stored as text |
| Top 2 orders per customer | sort_values then groupby().head(2) | ROW_NUMBER() <= 2 | Using nlargest on the whole frame |
import pandas as pd
df = pd.DataFrame({
'customer_id': ['C1', 'C1', 'C2', 'C2', 'C2'],
'order_date': pd.to_datetime(['2025-01-05', '2025-02-10',
'2025-01-02', '2025-01-20', '2025-03-01']),
'amount': [200, 800, 100, 100, 300]
})
# Rank within customer, biggest order = 1
df['rank_in_cust'] = (df.groupby('customer_id')['amount']
.rank(method='dense', ascending=False))
# Running total by date within customer
df = df.sort_values(['customer_id', 'order_date'])
df['running_spend'] = df.groupby('customer_id')['amount'].cumsum()
print(df)import pandas as pd
# Days since previous order (NaN for the first order)
df = df.sort_values(['customer_id', 'order_date'])
df['days_since_prev'] = (df.groupby('customer_id')['order_date']
.diff()
.dt.days)
# Average gap per customer
print(df.groupby('customer_id')['days_since_prev'].mean())
# Top 2 orders per customer
top2 = (df.sort_values('amount', ascending=False)
.groupby('customer_id')
.head(2)
.sort_values(['customer_id', 'amount'], ascending=[True, False]))
print(top2)
Notice that sorting comes first in three of the four answers. Running totals, gaps and top-N all depend on row order within each group. Saying 'I need to sort by date first' before writing the code is a clear sign of experience.
SECTION 04Practice set: group-to-row questions
Work through these until you can solve each one without notes and explain the output shape in one sentence. Use the sample data in the second tab so you can check every answer by hand.
GROUP-TO-ROW PRACTICE (one value per original row)
1. Share of each order in its customer's total
2. Share of each product in its category's sales
3. Difference of each salary from the dept average
4. Flag each student's best subject score
5. Count of orders per customer, on every row
6. Rank products by sales within each region
7. Running total of sales per region by month
8. Days since previous visit per patient
9. Top 3 products per category
10. Keep only cities with total sales above 10000
For each: agg, transform, rank, cumsum, diff,
head or filter?import pandas as pd
sales = pd.DataFrame({
'region': ['North', 'North', 'North', 'South', 'South'],
'category': ['Tech', 'Tech', 'Home', 'Tech', 'Home'],
'product': ['Laptop', 'Mouse', 'Lamp', 'Phone', 'Chair'],
'month': pd.to_datetime(['2025-01-01', '2025-02-01',
'2025-01-01', '2025-01-01', '2025-02-01']),
'sales': [50000, 2000, 3000, 30000, 7000]
})
# Q2 answer to check against:
sales['cat_share'] = (sales['sales'] /
sales.groupby('category')['sales'].transform('sum'))
print(sales)SAY THIS SHAPE SENTENCE FOR EACH ANSWER
'The input has one row per ______.
A plain aggregate would give one row per ______.
I need one value per ______, so I use ______.'
Then one check:
'To verify, I will ______
(e.g. confirm shares sum to 1 per group,
or that rank 1 is the largest amount).'
Record yourself. If the sentence is unclear,
the interviewer will find the code unclear too.
SECTION 05Habits that stop the group-total fumble
These habits help with the share question and with every other problem that mixes group values and row values.
- Ask: per group or per row? — Decide the output shape before choosing a method. Per group means agg; per row means transform, rank, cumsum or diff.
- Print the index of results — When an assignment produces NaN, print the index of both sides. Mismatched labels are almost always the reason.
- Sort before order-based logic — Running totals, gaps and top-N within groups depend on row order, so sort by the group and the date first.
- Verify with a group sum — For share calculations, check that each group's shares add to 1. It catches division by the wrong total.
- Link to SQL windows — Mentioning PARTITION BY shows the interviewer you understand the concept across tools, not just one function.
SECTION 06Interview Q&A — Pandas transform question
Q1What is the difference between agg and transform in groupby?
agg reduces each group to one row, so the output has one row per group with the keys in the index. transform also computes per group, but it returns a result aligned to the original rows, so every row gets its group's value. I use agg for summary tables and transform when I need a group value on each row, such as a share or a difference from the group mean.
Q2Why does assigning a groupby sum to a new column give NaN?
Because Pandas assigns by matching index labels. The groupby result is indexed by the group keys, like C1 and C2, while the DataFrame is indexed by 0, 1, 2 and so on. No labels match, so the new column is filled with NaN. transform or a merge fixes this.
Q3How do you rank rows within each group in Pandas?
Use df.groupby('group')['value'].rank(ascending=False). The method argument controls ties: 'min' gives tied values the same lowest rank, 'dense' leaves no gaps, and 'first' ranks by order of appearance. I choose based on what the business wants to see for ties.
Q4How do you calculate days since a customer's previous order?
Make sure the date column is a datetime, sort by customer and date, then use df.groupby('customer_id')['order_date'].diff().dt.days. The first order for each customer gets NaN because there is no previous order, which I would keep or label clearly.
Q5How would you get the top 2 orders for each customer?
Sort the whole DataFrame by amount in descending order, then use groupby('customer_id').head(2). This keeps the first two rows of each group after sorting. An alternative is to rank within groups and filter rank <= 2, which is useful when ties need special handling.
SECTION 07Test yourself — Pandas transform question quiz
Five questions. No sign-up.
0 / 5Pick an answer to see why it is right or wrong.
SECTION 08Frequently asked questions
What is groupby transform in Pandas?
It applies a function to each group and returns a result with the same shape as the original data, so each row gets the value for its group. Common uses are group totals, group means, counts and normalising values within groups.
How do I calculate percentage of total by group in Pandas?
Divide the value column by the group total using transform: df['pct'] = df['value'] / df.groupby('group')['value'].transform('sum') * 100. Check that the percentages add up to 100 within each group.
When should I use apply instead of transform in Pandas?
Use transform when you need one value per row based on the group. Use apply only when the logic is too complex for built-in methods, because apply is slower and its output shape can be harder to predict.
What Pandas groupby questions are asked in interviews?
Common ones include totals and averages per group, counting unique values, share of group total, ranking within groups, running totals, top-N per group, and filtering groups by a condition. Most test whether you understand output shapes.
Is merge or transform better for adding group totals?
Both give correct results. transform is shorter and keeps the original index, so it is usually preferred. Merge is useful when you are adding several aggregated columns at once or joining a summary table built elsewhere.
Is there a course that covers Pandas interview questions?
Yes. Uncodemy's Python Training Course in Noida (classroom and online) includes live projects, one-to-one mock interviews and resume reviews, so you practise Pandas interview questions with trainer feedback before the real interview. You can book a free demo class from this page to see how the sessions work.
SECTION 09Related reads
Classroom & online · Noida
Turn the group-total fumble into a strength
Our Python Training Course covers Pandas groupby, transform and window-style logic through hands-on datasets, so the questions most candidates fumble become routine for you.
Talk to a counsellor for current fees- Pandas problem practice
- SQL and Pandas side by side
- Live coding mock rounds
- Weekday & weekend batches