Why Business Analyst Candidates Can't Defend Their Numbers

DEFENDED NUMBER — FIRST 90 SECONDS:

Interviewer: "You say you cut rework 18%.
Walk me through that."

Candidate: "Sure. Let me define it and give
you the caveats before the number.

Rework here means a user story that was
reopened after development because the
acceptance criteria were unclear or
incomplete — not bugs in production, which is
a different metric.

With that definition: the baseline quarter
had 40 reopened stories out of 220 delivered
— 18.2%. The next quarter had 24 reopened out
of 214 — 11.2%. That's a 7-point drop, or 38%
relative. The '18%' I quoted was the drop in
reopened stories as a share of total, which
is a different calculation.

The attribution: most of the drop came from
adding acceptance criteria to 60 stories in
the checkout flow. I can't cleanly split it
from the sprint-boundary change we made at
the same time — they overlap, and I'd rather
report both than claim one.

The cost: writing acceptance criteria added
about 4 hours per story, so I wrote them for
60 high-value stories, not all 220. Coverage
is partial.

The caveat: the reopened count includes one
story that was reopened for a scope change,
not clarity — I left it in because removing
it would look like I'm cherry-picking.

I can show you the reopened-story log and the
criteria template."

What the interviewer thinks in 90 seconds:
- "They defined the metric first." ✅
- "They gave the baseline and the
  denominator." ✅
- "They split attribution honestly." ✅
- "They named the cost: partial coverage." ✅
- "They can show the evidence." ✅

Signal: defended. The number survives the
follow-up because it was defended before the
follow-up.

UNDEFENDED NUMBER — FIRST 90 SECONDS:

Interviewer: "You say you cut rework 18%.
Walk me through that."

Candidate: "Yeah, we reduced rework by 18%
because we improved our requirements process
and focused on quality. We wrote better user
stories and the team was more aligned, so the
number went down."

What the interviewer thinks in 90 seconds:
- "No definition — rework from where?" ❌
- "No baseline number — 18% of what?" ❌
- "No attribution — 'improved process' is
  not a cause." ❌
- "No cost or caveat." ❌
- "I'll ask a follow-up and the number will
  collapse." ⚠️

What happens next:
- The interviewer asks "What was the
  baseline reopened count?"
- The candidate says "I don't remember the
  exact number."
- The interviewer asks "Was any of the drop
  from a scope change?"
- The candidate pauses.

Signal: undefended. The number collapses
under one follow-up.

Why it failed:
❌ No metric definition
❌ No baseline number
❌ No attribution of causes
❌ No cost or caveat
❌ The first 90 seconds signaled "I read
  the slide," not "I measured this"

ATTRIBUTION ANSWER — WHAT DECIDES:

Interviewer: "How do you know your work
caused the drop?"

WHAT THE CANDIDATE SAYS:
"I don't claim all of it, and that's the
honest answer. Here's how I split it.

We tracked reopened stories by flow. The
checkout flow — where I wrote acceptance
criteria for 60 stories — went from 22
reopened to 8. The legacy billing flow went
from 12 to 10, but that's not my work: we cut
scope on legacy, so fewer stories shipped and
fewer could reopen. The API layer went from 6
to 6, which is noise.

So my criteria work contributed about 14 of
the 16 total drop. The scope cut removes 2
from the denominator entirely — those stories
never shipped, so they couldn't reopen.

I can't prove causation with one quarter of
data. What I can show is the flow breakdown,
the reopened log, and the fact that the
checkout drop started the sprint after the
criteria went in, not before.

The trade-off: writing criteria added 4 hours
per story, so I covered 60 of 220 stories,
not all of them. I chose the checkout flow
because that's where reopens cost the most —
a checkout defect blocks revenue.

I can show you the flow breakdown and the
criteria template."

What the interviewer thinks:
- "They split the drop by flow." ✅
- "They separated their work from the scope
  cut." ✅
- "They admitted they can't prove
  causation." ✅
- "They named the cost: 4 hours per
  story." ✅
- "They prioritized by business cost." ✅

Signal: confirmed defense.

Why it decides:
✅ Flow breakdown, not a total
✅ Scope cut separated out
✅ Causation caveat stated
✅ Cost is quantified
✅ Prioritization is business-based

COVERAGE ANSWER — WHAT DECIDES:

Interviewer: "You said you covered 60
stories. Out of how many?"

WHAT THE CANDIDATE SAYS:
"60 out of 220 — about 27% coverage, and I'll
be honest about why that's not higher.

The reason: writing a good acceptance
criteria set takes about 4 hours per story —
you have to talk to the developer, the
tester, and often the stakeholder. For 220
stories, that's 880 hours, which is half a
person-year. I didn't have that.

So I prioritized. I picked the 60 stories in
the checkout flow, because that's where a
reopened story costs the most — a checkout
defect blocks revenue, while a reporting
defect delays a chart. The rework number
dropped 7 points overall, but in checkout it
dropped 14 points.

The caveat: the 60 stories are the ones with
the highest reopen rate historically, so the
coverage number looks better than a random
sample would. That's a selection bias I'd
flag — if you asked me to prove the criteria
work across all flows, this data doesn't.

The next step: I documented the criteria
template so other BAs can apply it to their
flows without me. That's the scaling path —
not me writing 220 sets, but the team writing
them with a template.

The trade-off: the template needs a review
cadence, or the criteria drift. I set up a
30-minute weekly review for the first
quarter; after that, the team owns it.

I can show you the template and the
prioritization rationale."

What the interviewer thinks:
- "They named the coverage: 27%." ✅
- "They explained the cost: 4 hours per
  story." ✅
- "They prioritized by business cost." ✅
- "They flagged the selection bias." ✅
- "They named the scaling path." ✅

Signal: confirmed defense.

Why it decides:
✅ Coverage is quantified
✅ Cost is explained
✅ Prioritization is business-based
✅ Selection bias is flagged
✅ Scaling path is named

CYCLE TIME — DEFENDED VS. UNDEFENDED:

Scenario: "You say you cut claims cycle time
30%. Walk me through it."

UNDEFENDED SIGNAL:
"We improved the claims process and cut
cycle time by 30%. The team was more
efficient and the stakeholders were happy."
What the interviewer thinks:
- "30% of what — days, hours, business
  days?"
- "From what baseline?"
- "What did you change?"
- Signal: undefended.

DEFENDED SIGNAL:
"Let me define it first, because cycle time
has two common definitions and they give
different numbers.

Definition here: claims cycle time is the
calendar days from claim intake to payment
decision — not from intake to first review,
which would be shorter. With that definition,
the baseline was 4.2 days and we got it to
2.9 days — a 31% drop. I said 30% because
it's rounder, but 31% is the number.

The change: I mapped the exception flow in
BPMN and found 3 handoff gaps — points where
a claim sat in a queue waiting for someone to
notice it. The gaps accounted for 1.1 of the
1.3-day drop. The remaining 0.2 came from a
new intake form that caught missing
documents earlier.

The attribution: the BPMN mapping and the
intake form were both my work, but I
wouldn't claim them as one cause. The 1.1
days is the handoff fix; the 0.2 is the form.
They're additive here because they don't
overlap — but I'd want three more months of
data to be confident.

The cost: the intake form added a required
field that takes the claimant about 30
seconds to fill. The trade-off is 30 seconds
of claimant time for a 0.2-day reduction in
cycle — worth it, but worth naming.

The caveat: the 2.9 days is an average. The
p90 is 6.1 days, driven by a compliance
review that's outside my scope. If you want
to improve the p90, that review is the
target.

I can show you the BPMN map and the
handoff list."

What the interviewer thinks:
- "They defined the metric first." ✅
- "They gave the baseline and the
  denominator." ✅
- "They split the attribution: 1.1 and
  0.2." ✅
- "They named the cost: 30 seconds." ✅
- "They named the caveat: p90." ✅
- Signal: confirmed defense.

Why it decides:
✅ Definition first
✅ Baseline and denominator
✅ Attribution split
✅ Cost named
✅ Caveat named

INTERVIEW FOLLOW-UP — DEFENDED VS.
UNDEFENDED:

Scenario: "You said you aligned 4
stakeholders. How did you know they were
aligned?"

UNDEFENDED SIGNAL:
"We had meetings and everyone agreed. The
stakeholders were on the same page after
the meetings."
What the interviewer thinks:
- "Agreement in a meeting isn't alignment."
- "What did they agree on?"
- "Did they act on it?"
- Signal: undefended.

DEFENDED SIGNAL:
"Three pieces of evidence, and one caveat.

Evidence one: a shared definition of done.
Before the work, each stakeholder had their
own definition — product wanted 'feature
complete,' compliance wanted 'all documents
verified,' engineering wanted 'tests passing.'
I ran a 3-week discovery with 5 interviews
and got the four of them into one room to
write a single definition. That definition is
in the story template — 60 stories use it.

Evidence two: the rework number. When
stakeholders disagree on done, stories get
reopened — the team builds to one definition
and gets rejected against another. Rework
dropped from 40 reopened to 24 in the
quarter after the definition went in. That's
evidence they're acting on the shared
definition, not just agreeing in a meeting.

Evidence three: a conflict they resolved. In
week 3 of the discovery, compliance had a
hard requirement that wasn't in the original
scope — a document verification step. That
requirement added 12 acceptance criteria.
The stakeholders agreed to it in the
definition session, not by email later. That's
the strongest evidence — a real conflict,
resolved in the room.

The caveat: I can't prove they're still
aligned six months later. Alignment drifts
when the stakeholders change. I'd want a
quarterly review of the definition to keep it
current.

The cost: the definition session was 4 hours
of four senior people's time. That's
expensive — about $2K in loaded cost — but
the alternative was 40 reopened stories a
quarter, which is more.

I can show you the definition document and
the story template."

What the interviewer thinks:
- "Three pieces of evidence." ✅
- "The rework number is behavioral
  evidence." ✅
- "The conflict resolution is the strongest
  evidence." ✅
- "The caveat is honest." ✅
- "The cost is quantified: $2K." ✅
- Signal: confirmed defense.

Why it decides:
✅ Three pieces of evidence
✅ Behavioral evidence, not just words
✅ Conflict resolution named
✅ Caveat is honest
✅ Cost is quantified

BA Number Defense Checklist:

FIRST 90 SECONDS:
☐ Define the metric: what counts, what
  doesn't
☐ State the baseline number and the time
  window
☐ State the denominator: stories, releases,
  claims
☐ Name one confounder or cost early
☐ Avoid "we improved the process" as a
  cause

WHAT DECIDES:
☐ Attribute the change to specific causes
☐ Separate your work from scope cuts and
  tooling changes
☐ Name the cost: hours, coverage, meetings
☐ Name the caveats: what the metric
  doesn't measure
☐ Name the coverage: 60 of 220, not "all"

WHAT CONFIRMS:
☐ Show the evidence: log, report, timing
☐ Admit where causation can't be proven
☐ Name what you'd do differently
☐ Answer "what was the baseline?" without
  pausing
☐ Answer "what's the catch?" without
  hedging

RED FLAGS:
❌ Quotes a percentage with no baseline
❌ "We improved the process" as a cause
❌ No denominator
❌ No cost or caveat
❌ Can't show the log or report
❌ "All stories" when it was 60 of 220
❌ Can't answer one follow-up

Why this works:
- The first 90 seconds set the signal
- The rest of the interview confirms or
  overturns it
- Baselines, attribution, and caveats are
  the three things that can't be faked

30-DAY PRACTICE PLAN (FOR BAS):

Week 1 — Metric definitions
☐ Pick 5 metrics from your resume
☐ For each, write the definition and the
  baseline number
☐ Write the denominator
☐ Record yourself defending each in
  90 seconds
☐ Cut every "improved process" sentence

Week 2 — Attribution
☐ For each metric, list the causes of the
  change
☐ Separate your work from scope and
  tooling wins
☐ Write the split as percentages or counts
☐ Name the cost of each change

Week 3 — Caveats
☐ For each metric, write what it doesn't
  measure
☐ Name the coverage: 60 of 220
☐ Write a "what I'd do differently" line
☐ Add these to your interview notes

Week 4 — Mock defense
☐ Run 2 mock interviews with a peer
☐ Ask them to challenge every number with
  one follow-up
☐ Re-run with stronger baselines and
  caveats
☐ Compare the two recordings

Signs it's working:
✅ You define the metric before quoting it
✅ You state the baseline without pausing
✅ You separate your work from scope
  cuts
✅ You name the cost and the caveat
✅ You name the coverage
✅ The interviewer asks "what else?" instead
  of "what was the baseline?"