Why Data Scientists Who Ask Questions Back Impress Me Most

Interviewer: We want to predict churn for our app.

Candidate: Sure. I would collect user features like age,
sessions and purchases, handle missing values, then try
logistic regression, random forest and XGBoost, tune them
with grid search and pick the best AUC.

Interviewer: What is your target variable?

Candidate: Churn... whether the user churned.

Interviewer: We want to predict churn for our app.

Candidate: Before I design anything, three quick questions.

First, is churn a cancelled paid plan, or users who stop
opening the app? Those are different problems.

Second, how far ahead do you need to know? A week ahead
supports a quick nudge; a quarter ahead supports pricing.

Third, what happens to flagged users? A discount, a call,
an email? That tells me how costly false alarms are.

Interviewer: Paid cancellations, 30 days ahead, and we
send an in-app offer.

Candidate: Great. Then the label is 'cancelled within
30 days of the snapshot date', and because an in-app offer
is cheap, I can favour recall over precision.

-- PostgreSQL. Label: was a paid user at the snapshot date
-- still subscribed 30 days later? 1 = churned.
WITH snapshot AS (
    SELECT DISTINCT user_id, DATE '2024-06-30' AS snapshot_date
    FROM subscriptions
    WHERE start_date <= DATE '2024-06-30'
      AND (end_date IS NULL OR end_date >= DATE '2024-06-30')
)
SELECT s.user_id,
       s.snapshot_date,
       CASE WHEN EXISTS (
                SELECT 1
                FROM subscriptions AS x
                WHERE x.user_id = s.user_id
                  AND x.start_date <= s.snapshot_date + 30
                  AND (x.end_date IS NULL
                       OR x.end_date >= s.snapshot_date + 30)
            )
            THEN 0 ELSE 1 END AS churned_30d
FROM snapshot AS s;

import pandas as pd

# events: user_id, session_id, event_ts
ref = pd.Timestamp("2024-06-30")
events["event_date"] = pd.to_datetime(events["event_ts"]).dt.normalize()

last_7 = events[(events["event_date"] > ref - pd.Timedelta(days=7)) &
                (events["event_date"] <= ref)]

sessions = last_7.groupby("user_id")["session_id"].nunique()
active = sessions[sessions >= 3].index

print(len(active), "active users in the week ending", ref.date())

# Say aloud: "I count distinct sessions, not events, so one
# long session full of clicks does not make someone active."

RECOMMENDATIONS
- What is recommended: products, articles, people?
- Where is it shown, and how many slots?
- How do we handle new users with no history?

FORECASTING
- What level: store, product, or store-product?
- How far ahead, and how often is it refreshed?
- Which decisions use it: ordering, staffing?

FRAUD
- How long until a fraud label is confirmed?
- Block, review, or only flag?
- Is a human reviewer available, and how many cases a day?

PRICING
- Can prices change per user, or only per segment?
- Are there legal or fairness limits?

HOW MANY QUESTIONS IS ENOUGH?

Aim for two to four focused questions, then commit.

If the interviewer says 'you decide', do not stall.
State an assumption and move on:

"Then I will assume churn means a cancelled paid plan,
 predicted 30 days ahead, with a cheap in-app offer.
 If any of that is wrong, the label and threshold change,
 but the overall approach stays the same."

This shows you can work with uncertainty, which is the
normal state of real projects.

PROMPT: __________________________________

Target: what exactly is predicted? ______________
Horizon: how far ahead? _________________________
Action: who uses it and what do they do? ________
Data: what is known at prediction time? _________
Constraint: speed, explanation, budget? _________

Pick the 3 questions whose answers would change
your design the most. Ask only those first.

USE ONE PER DAY

1. Predict which delivery orders will be late.
2. Recommend courses to learners on an edtech app.
3. Detect fake reviews on a shopping site.
4. Forecast daily demand for a grocery chain.
5. Predict which leads a sales team should call first.
6. Flag suspicious insurance claims.
7. Estimate the price of a used car from a listing.

For each: ask 3 questions, then state assumptions
for anything left unanswered.

PARTNER SCORES EACH ATTEMPT (Y / N)

[ ] Asked about the target definition
[ ] Asked about the horizon or timing
[ ] Asked what action follows the prediction
[ ] Asked at least one constraint question
[ ] Asked no more than five questions before starting
[ ] Turned each answer into a stated design choice
[ ] Stated assumptions when an answer was refused

Target: all seven ticked by the end of the week.