Why Data Engineer Candidates Can't Defend Their Own Numbers

DEFENDED NUMBER — FIRST 90 SECONDS:

Interviewer: "You say you cut pipeline cost
32%. Walk me through that."

Candidate: "Sure. Let me define it and give
you the caveats before the number.

Pipeline cost here means the monthly bill for
the ETL cluster plus the S3 storage the
pipeline writes — not the whole cloud bill,
which includes the app servers and the
database.

With that definition: the baseline quarter
was $44K a month. The next quarter was $30K.
That's a 32% drop. Same data volume — 12M
rows a day — so the comparison holds.

The attribution: three changes, and I can
claim two of them.

One: the spot move. The ETL ran on on-demand
for 6 hours a night. I moved it to spot with
a fallback, which saved about $9K a month.

Two: rightsizing. I pulled 30 days of metrics
and found 40 instances under 15% CPU. I
downsized them, which saved about $3K.

Three: a vendor discount. The cloud provider
gave us a committed-use discount that I didn't
negotiate — that's $2K. I can't claim it.

The cost: the spot move means the ETL is
occasionally 20 minutes late. The SLA is 6 AM
and it finishes by 4, so we accepted it. I
also added a fallback to on-demand after 3
minutes if the spot is reclaimed.

I can show you the cost breakdown and the
fallback config."

What the interviewer thinks in 90 seconds:
- "They defined the metric first." ✅
- "They gave the baseline and the
  denominator." ✅
- "They split attribution and disclaimed
  the vendor discount." ✅
- "They named the cost: 20 minutes late." ✅
- "They can show the evidence." ✅

Signal: defended. The number survives the
follow-up because it was defended before the
follow-up.

UNDEFENDED NUMBER — FIRST 90 SECONDS:

Interviewer: "You say you cut pipeline cost
32%. Walk me through that."

Candidate: "Yeah, we reduced the pipeline
cost by 32% because we optimized the
infrastructure and moved to cloud-native
services. The team did a great job and the
bill went down."

What the interviewer thinks in 90 seconds:
- "No definition — cost from where?" ❌
- "No baseline — 32% of what?" ❌
- "No attribution — 'optimized the
  infrastructure' is not a cause." ❌
- "No cost or caveat." ❌
- "I'll ask a follow-up and the number will
  collapse." ⚠️

What happens next:
- The interviewer asks "What was the
  baseline bill?"
- The candidate says "I don't remember the
  exact number."
- The interviewer asks "Was any of the drop
  from a vendor discount?"
- The candidate pauses.

Signal: undefended. The number collapses
under one follow-up.

Why it failed:
❌ No metric definition
❌ No baseline number
❌ No attribution of causes
❌ No cost or caveat
❌ The first 90 seconds signaled "I read
  the slide," not "I measured this"

ATTRIBUTION ANSWER — WHAT DECIDES:

Interviewer: "How do you know your work caused
the drop?"

WHAT THE CANDIDATE SAYS:
"I don't claim all of it, and that's the
honest answer. Here's how I split it.

The $14K drop breaks into three parts:

One: the spot move. The ETL ran on on-demand
for 6 hours a night — $4K a month. Spot is
70% cheaper, so it's $1.2K. That's $2.8K of
the drop, not $9K. Let me correct myself —
the $9K figure includes the fallback overhead
and the savings from the other two changes.

Two: rightsizing. 40 instances under 15% CPU,
downsized — that's $2K a month.

Three: the vendor discount. The cloud provider
gave us a committed-use discount that sales
negotiated — that's $2K a month. I didn't
negotiate it, so I can't claim it.

So my work is about $4.8K of the $14K drop —
roughly a third. The rest is the vendor
discount and a scope change that reduced the
data volume by 8%.

The scope change: we dropped the legacy
reporting pipeline in Q2, which cut the
volume. That's a business decision I didn't
make — I just implemented it. It removes
about $4K from the denominator.

The trade-off: the spot move adds a fallback
that pages on-call if the spot is reclaimed
twice in a week. It's paged twice since
launch. I accepted that because 20 minutes of
ETL delay is cheaper than $2.8K a month.

The lesson: I now write the attribution before
I quote a number. If I can't split it, I
shouldn't claim it."

What the interviewer thinks:
- "They corrected their own number." ✅
- "They split the drop into three parts." ✅
- "They disclaimed the vendor discount." ✅
- "They separated their work from a scope
  cut." ✅
- "They admitted the pager." ✅

Signal: confirmed defense.

Why it decides:
✅ Self-correction on the number
✅ Attribution split into parts
✅ Vendor discount disclaimed
✅ Scope cut separated out
✅ The cost is named: two pages

CAVEAT ANSWER — WHAT DECIDES:

Interviewer: "What's the catch?"

WHAT THE CANDIDATE SAYS:
"Three catches, and I'll name each.

Catch one: the spot fallback. The ETL is
occasionally 20 minutes late when the spot is
reclaimed. The SLA is 6 AM and it finishes by
4, so we accepted it. I added a fallback to
on-demand after 3 minutes — the cost is a
pager if it fires twice in a week. It's fired
twice since launch.

Catch two: the rightsizing risk. I downsized
40 instances based on 30 days of metrics. If
traffic spikes 3x, those instances will
saturate. I added an autoscaling group that
scales past the old size if CPU crosses 70% —
but the scale-out takes about 90 seconds,
which is a real window of risk.

Catch three: the vendor discount. It expires
in 14 months. When it expires, the pipeline
cost goes up $2K a month unless we renegotiate.
I've flagged it to the finance team — it's on
their calendar, not mine.

The cost I accepted: the fallback pager is
noise at 3 AM. I took it over the alternative
— a partial batch, which would corrupt the
report.

The trade-off I'd revisit: the autoscaling
group doubles the instance count at peak, so
a spike costs more than the steady state. If
we had predictable traffic, I'd use a reserved
instance instead.

The lesson: a cost number has a shelf life.
The vendor discount expires, traffic grows,
and the number moves. I'd re-measure every
quarter, not once."

What the interviewer thinks:
- "Three catches, each named." ✅
- "The spot pager is quantified: two
  fires." ✅
- "The rightsizing risk has a mitigation." ✅
- "The vendor discount has an expiry date." ✅
- "The shelf life is named." ✅

Signal: confirmed defense.

Why it decides:
✅ Three catches, each specific
✅ The pager is quantified
✅ The mitigation is named
✅ The expiry is named
✅ The shelf life is honest

LATENCY — DEFENDED VS. UNDEFENDED:

Scenario: "You say you cut pipeline latency
60%. Walk me through it."

UNDEFENDED SIGNAL:
"We improved the pipeline and latency went
down 60%. The team did a great job optimizing
the queries."
What the interviewer thinks:
- "60% of what — p50, p99, end-to-end?"
- "From what baseline?"
- "What did you change?"
- Signal: undefended.

DEFENDED SIGNAL:
"Let me define it first, because 'latency'
has two common definitions and they give
different numbers.

Definition here: pipeline latency is the
end-to-end time from the last event landing
in Kafka to the row being queryable in the
warehouse — not the query time, which would
be shorter. With that definition, the baseline
was 4.2 seconds at p99 and we got it to 1.6
seconds — a 62% drop. I said 60% because it's
rounder, but 62% is the number.

The change: I found a fanout in the join
between events and the account dimension. A
duplicate account key duplicated every event
for that account — about 12% of events. The
fix was a dedup step before the join, which
took 2.4 seconds off the p99. The remaining
0.2 seconds came from a Redis cache on the
account lookup.

The attribution: the fanout fix is 2.4
seconds; the cache is 0.2. They're additive
here because they don't overlap — but I'd
want three more months of data to be
confident.

The cost: the cache adds a stale window of 5
minutes for account changes. The trade-off is
5 minutes of staleness for 0.2 seconds of
latency — worth it for the dashboard, but I
wouldn't use it for the compliance report.

The caveat: the 1.6s is an average. The p99.9
is still 6 seconds, driven by a backfill job
that's outside the streaming path. If you
want to improve the p99.9, that backfill is
the target.

I can show you the fanout query and the cache
config."

What the interviewer thinks:
- "They defined the metric first." ✅
- "They gave the baseline and the
  denominator." ✅
- "They split the attribution: 2.4 and
  0.2." ✅
- "They named the cost: 5 minutes of
  staleness." ✅
- "They named the caveat: p99.9." ✅
- Signal: confirmed defense.

Why it decides:
✅ Definition first
✅ Baseline and denominator
✅ Attribution split
✅ Cost named
✅ Caveat named

INTERVIEW FOLLOW-UP — DEFENDED VS.
UNDEFENDED:

Scenario: "You said you reduced failed runs.
How did you know the fix worked?"

UNDEFENDED SIGNAL:
"I ran the pipeline and it didn't fail as
much. The team was happy."
What the interviewer thinks:
- "That's not a measurement, that's a hope."
- "No baseline, no comparison."
- Signal: undefended.

DEFENDED SIGNAL:
"Three checks, and I can show the numbers.

Check one: I compared the failure rate
before and after the fix on the same volume.
Before: 8 failed runs a week on 12M rows a
day. After: 1 failed run a week. Same volume,
so the comparison holds.

Check two: I broke down the failures by
cause. 6 of the 8 were the missing retry on
the S3 upload. 1 was a timeout on the
warehouse load. 1 was an upstream schema
change. The retry fixed the 6; the timeout
is still there — I flagged it as a separate
ticket. So my fix addressed 75% of the
failures, not 100%.

Check three: I back-tested the retry against
a month of history. It would have caught 5 of
the 6 S3 failures — the sixth was a permission
issue the retry can't fix. So the honest
number is 5 of 6, not 6 of 6.

The cost: the retry adds 2 minutes to the
failure path. I chose that over faster
failures because a 2-minute delay is cheaper
than a 6-hour manual reprocess.

The lesson: I now break failures down by
cause before I quote a number. '8 to 1' hides
the fact that 2 failures are still there and
one is out of scope."

What the interviewer thinks:
- "Three checks with numbers." ✅
- "The breakdown by cause is specific." ✅
- "They admitted the fix is 75%, not
  100%." ✅
- "The back-test is evidence." ✅
- "The cost is quantified: 2 minutes." ✅
- Signal: confirmed defense.

Why it decides:
✅ Baseline and after, same volume
✅ Breakdown by cause
✅ The fix is 75%, not 100%
✅ The back-test is real
✅ The cost is quantified

Data Engineer Number Defense Checklist:

FIRST 90 SECONDS:
☐ Define the metric: what's in, what's out
☐ State the baseline number and the time
  window
☐ State the denominator: rows, volume,
  requests
☐ Name one confounder or cost early
☐ Avoid "we optimized the infrastructure" as
  a cause

WHAT DECIDES:
☐ Attribute the change to specific causes
☐ Disclaim the wins you didn't cause: the
  vendor discount, the scope cut
☐ Name the cost: 20 minutes late, 5 minutes
  stale, 2 minutes per failure
☐ Name the caveats: what the metric doesn't
  measure
☐ Name the fix percentage: 75%, not 100%

WHAT CONFIRMS:
☐ Show the evidence: log, dashboard,
  back-test
☐ Admit where causation can't be proven
☐ Name what you'd do differently
☐ Answer "what was the baseline?" without
  pausing
☐ Answer "what's the catch?" without
  hedging

RED FLAGS:
❌ Quotes a percentage with no baseline
❌ "We optimized the infrastructure" as a
  cause
❌ No denominator
❌ No cost or caveat
❌ Can't show the log or dashboard
❌ "The team was happy" as evidence
❌ Can't answer one follow-up

Why this works:
- The first 90 seconds set the signal
- The rest of the interview confirms or
  overturns it
- Baselines, attribution, and caveats are
  the three things that can't be faked

30-DAY PRACTICE PLAN (FOR DATA
ENGINEERS):

Week 1 — Metric definitions
☐ Pick 5 metrics from your resume
☐ For each, write the definition and the
  baseline number
☐ Write the denominator
☐ Record yourself defending each in
  90 seconds
☐ Cut every "optimized the infrastructure"
  sentence

Week 2 — Attribution
☐ For each metric, list the causes of the
  change
☐ Separate your work from vendor discounts
  and scope cuts
☐ Write the split as percentages or counts
☐ Name the cost of each change

Week 3 — Caveats
☐ For each metric, write what it doesn't
  measure
☐ Name the fix percentage: 75%, not 100%
☐ Write a "what I'd do differently" line
☐ Add these to your interview notes

Week 4 — Mock defense
☐ Run 2 mock interviews with a peer
☐ Ask them to challenge every number with
  one follow-up
☐ Re-run with stronger baselines and
  caveats
☐ Compare the two recordings

Signs it's working:
✅ You define the metric before quoting it
✅ You state the baseline without pausing
✅ You disclaim the vendor discount
✅ You name the cost and the caveat
✅ You name the fix percentage
✅ The interviewer asks "what else?" instead
  of "what was the baseline?"