Why Every ML Engineer Should Practise Explaining Their Test Case Aloud

Career Guide · ML Engineering

Why Every ML Engineer Should Practise Explaining Their Test Case Aloud

SECTION 01Why aloud, not silent

Reading your test case silently is fast. Speaking it is slow. That slowness is where the practice happens.

Say it aloud and you'll hear yourself say things like:

• "Model got 0.87 AUC on… a holdout…" — no split, no baseline.
• "We used a random 80/20 split… I think…" — hedging on data leakage.
• "It generalises well…" — assumption, not evidence.

Each stumble is a place an interviewer would push. Fixing it at home saves you from being asked about it live.

SECTION 02The 3 levels of explanation

Level 1 — Read it

"Model got 0.87 AUC on a holdout set." This describes the metric. Anyone could say this.

Level 2 — Explain it

"Time-based split — last 90 days as holdout — so no user appears in both sets. Class balance held at 4.2%. Baseline was a rule engine at 0.81 AUC." This shows evaluation rigour.

"Model got 0.87 AUC on a holdout set."

"Time-based split — last 90 days as holdout —
so no user appears in both sets. Class balance
held at 4.2% in both splits. Baseline was the
current rule engine at 0.81 AUC. We evaluated
on 3 metrics (AUC, precision-at-recall-80,
calibration) because the business cares about
false positives more than raw AUC."

Level 3 — Defend it

"Yes, there was a leakage risk — some users appeared in both train and test. We dropped them and AUC held at 0.85, so the lift is real, not leakage." This shows scientific maturity.

Interviewer: "Could a user be in both train
and test?"

"Good question — yes, initially. We caught it
during a sanity check: 8% of test users had
also appeared in training. We dropped those
users and re-ran: AUC fell from 0.87 to 0.85
and the precision-at-recall-80 held at 0.62.
So the lift over baseline is real, not leakage.
We now enforce the no-overlap constraint in the
feature pipeline so future runs can't leak."

SECTION 03The 4-step routine

Run this routine once per test case. It takes 8-10 minutes with a friend, or 15 minutes if you record and score it yourself.

Step 1 — Dataset (30 s)
"What data did you test on? How many rows?
 What class balance? What time window?"

Step 2 — Split (60 s)
"How did you split train/test? Random or
 time-based? Why? Name the one leakage path
 you eliminated."

Step 3 — Baseline (60 s)
"What's the baseline? Rule engine, logistic
 regression, previous model? What does your
 model do better than it?"

Step 4 — Metric (30 s)
"Which metric matters here, and why? AUC,
 precision@recall, calibration, latency?
 What's the business impact of the lift?"

Total: 3 minutes
Target: 3 minutes in an interview.

If you're under 90 seconds, you're describing
the metric, not the evaluation.

If you're over 5 minutes, you're reading the
code line-by-line — not the point.

Practice twice per test case:
  Pass 1: run through the 4 steps once.
  Pass 2: record, listen back, score it.

SECTION 04Common stumbles

When you say it aloud, these phrases come out. Each one signals a weakness the interviewer would have found anyway.

SECTION 05A self-scoring rubric

Record yourself explaining the test case. Play it back once. Score each line 0-2. A total of 8+/10 means you're interview-ready.

Scoring rubric (max 10):

☐ Dataset + class balance named (0-2)
☐ Split type + reason named (0-2)
☐ Baseline named with metric (0-2)
☐ Metric choice justified (0-2)
☐ Leakage check + result (0-2)

Bonus: handles one follow-up cleanly (+1)

Score guide:
  9-10 — interview-ready, can defend live
  6-8  — solid, needs one more pass
  3-5  — reading the metric, not the eval
  0-2  — you don't own this test case yet

SECTION 06Interview Q&A

SECTION 07Test yourself

SECTION 08Frequently asked questions

SECTION 09Related reads