Why Some DevOps Engineer Candidates Ace Every Round Except the Last

Manager: Tell me about a time production went down.

Candidate: The pod was in CrashLoopBackOff. I ran kubectl
describe and kubectl logs. There was a config error.
I fixed the ConfigMap and redeployed. It worked.

(Technically fine. But who did you inform? How long were
users affected? What stopped it happening again?
The manager learned nothing about you as a teammate.)

Manager: Tell me about a time production went down.

Candidate: In my project team, a release broke login.
I posted in the team channel that I was looking at it,
so nobody duplicated work. Pods were crash-looping, and
the previous logs showed a missing environment variable.

I rolled back first, because restoring login mattered
more than finding the perfect fix. Then I told everyone
it was back and what I suspected.

Afterwards we added a startup check for required
variables to the pipeline, and I wrote a short review
focused on the gap, not on who made the change.

Manager: A developer keeps skipping the pipeline and
deploying from their laptop. What would you do?

Candidate: First I would talk to them privately and ask
why. Often it is because the pipeline is slow or fails
for reasons unrelated to their change.

If that is true, I would fix that first: cache
dependencies, split slow tests, and give clear error
messages.

Then I would remove the need for laptop deploys by
making production credentials available only to the
pipeline role. Both together usually solve it without
anyone feeling blamed.

# GitHub CLI: require reviews and passing checks on main
gh api -X PUT repos/OWNER/REPO/branches/main/protection \
  -H "Accept: application/vnd.github+json" \
  --input - <<'EOF'
{
  "required_status_checks": {"strict": true, "contexts": ["build", "test"]},
  "enforce_admins": false,
  "required_pull_request_reviews": {"required_approving_review_count": 1},
  "restrictions": null
}
EOF

INCIDENT REVIEW (blameless)

Summary:      Login failed after release; rolled back.
Impact:       Users could not log in during the incident.
Timeline:     Detected -> acknowledged -> rolled back
              -> confirmed recovery (with timestamps)
Root cause:   Required env variable missing in new config.
Why it passed: Pipeline had no check for required vars.
What went well: Fast rollback, clear updates in channel.
Actions:
  [ ] Add startup check for required variables (owner)
  [ ] Add variable list to deployment checklist (owner)
  [ ] Alert on login error rate (owner)

Manager: Are you comfortable being on-call?

Candidate: Yes. I would like to shadow someone on the
rotation first, read the runbooks, and know exactly when
to escalate and to whom.

I also care about alert quality. If an alert wakes
someone up and needs no action, I would want to fix or
remove it.

Can I ask how the rotation works here and how alerts
are reviewed?

PREPARE ONE STORY EACH

1. Something broke and I fixed it
   Role: ________  First action: __________
2. I disagreed with a teammate or developer
   Their reason: ______  Our agreement: ______
3. I made a mistake
   What it broke: ______  What I changed: ______
4. I automated a boring task
   Before: ______  After: ______
5. I learned a tool quickly for a need
   Why: ______  How: ______

FOR EVERY STORY

Context   one sentence: where, what system
My role   what I personally owned
First     what I did first and why
People    who I told or worked with
Result    what changed (no invented numbers)
After     what I changed to prevent repeats

Check: do I say 'I' for my actions,
and 'we' only for real team decisions?

QUESTIONS FOR THE HIRING MANAGER

- How does a change reach production here?
- How does the on-call rotation work?
- How are incidents reviewed after they happen?
- What is the biggest source of toil for the team?
- How do developers and DevOps decide on new rules?
- What would a good first few months look like?

Pick two or three. Listen to the answers.