What a Great Pipeline Walkthrough Actually Sounds Like

Interview Insights · Data Analytics

What a Great Pipeline Walkthrough Actually Sounds Like

SECTION 01Follow one record from birth to dashboard

The simplest way to make a pipeline understandable is to pick a single record and follow it. For a bike-rental project, that record is one trip. It is born when a rider ends a ride and the operator's system records the start station, end station and times. My pipeline pulls it from the operator's API, lands it unchanged in a raw table, cleans and joins it with weather data in a daily model, and the operations team sees it as part of station demand on a dashboard at 8 am.

That paragraph is the whole walkthrough in miniature. It names the source, the landing zone, the transformation, the destination and the user. Everything else is detail, and the interviewer can now choose which detail to explore. Compare that with opening on 'I used Airflow, Python and PostgreSQL', which tells them nothing about what the system does.

"Let me follow one bike trip through the pipeline.

When a rider ends a trip, the operator's system records
it. Every night at 1 am my pipeline calls their API for
the previous day's trips and stores the response as-is
in a raw table, with the time we loaded it.

A SQL step then cleans the trips, removes rides shorter
than a minute, which are usually unlock errors, and joins
daily weather. The result is one row per station per hour.

The operations team opens a dashboard at 8 am to plan
where to move bikes."

import json
import requests
from datetime import date

def fetch_trips(api_base, api_key, day: date):
    resp = requests.get(
        f'{api_base}/trips',
        params={'date': day.isoformat()},
        headers={'Authorization': f'Bearer {api_key}'},
        timeout=30,
    )
    resp.raise_for_status()   # fail loudly on HTTP errors
    return resp.json()['trips']

def land_raw(conn, day, trips):
    with conn.cursor() as cur:
        cur.execute('DELETE FROM raw_trips WHERE trip_date = %s', (day,))
        cur.executemany(
            'INSERT INTO raw_trips (trip_date, payload, loaded_at) '
            'VALUES (%s, %s, now())',
            [(day, json.dumps(t)) for t in trips],
        )
    conn.commit()

Two design details are hidden in that extract code, and both are worth saying aloud: the raw payload is stored unchanged, so you can rebuild the model later if the cleaning logic changes, and the date's rows are deleted before insert, so rerunning a night is safe. The API base address and key come from configuration, not from the code.

SECTION 02Explain the orchestration: tasks, order, schedule and retries

Once the journey is clear, the next layer is orchestration: how the steps run in the right order, at the right time, and what happens when one fails. In Airflow this is a DAG, a directed acyclic graph of tasks. You do not need to read the code aloud. You need to explain four things: what the tasks are, which depends on which, when it runs and how failures are retried.

A great walkthrough also explains why the dependencies matter. The weather join must wait until both trips and weather have landed, otherwise the model is built on half the data. Retries handle temporary problems such as an API timeout, but a retry should never create duplicates, which is why each load step is rerun-safe. That connection between retries and idempotent loads is something interviewers listen for.

from datetime import datetime, timedelta
from airflow.decorators import dag, task

default_args = {
    'retries': 3,
    'retry_delay': timedelta(minutes=10),
}

@dag(
    schedule='0 1 * * *',          # daily at 1 am
    start_date=datetime(2026, 1, 1),
    catchup=False,
    default_args=default_args,
    tags=['bike-rentals'],
)
def bike_rentals():

    @task
    def extract_trips(ds=None):
        ...  # fetch_trips + land_raw for date ds

    @task
    def extract_weather(ds=None):
        ...  # same pattern for the weather API

    @task
    def build_model(ds=None):
        ...  # run SQL that rebuilds daily_station_demand for ds

    @task
    def check_freshness(ds=None):
        ...  # fail if the model has no rows for ds

    model = build_model()
    [extract_trips(), extract_weather()] >> model >> check_freshness()

bike_rentals()

"The DAG has four tasks. The two extracts run in
parallel, because trips and weather do not depend on
each other.

Building the model waits for both, so we never join trips
to missing weather. After that, a freshness check fails
the run if the model has no rows for the day.

Each task retries three times, ten minutes apart, which
covers most API timeouts. Because every load deletes and
reinserts its day, a retry cannot create duplicates.
If all retries fail, Airflow marks the run failed and
sends an email to me."

Airflow passes the run's logical date to tasks as ds, which is how each task knows which day to process. Mentioning that is a small sign you have actually run a DAG, because it explains how backfilling a missed day works: you trigger the DAG for that date, and every task processes that date only.

SECTION 03Close on freshness and the trade-offs you chose

A walkthrough that ends with 'and then it shows on the dashboard' feels unfinished. A great one closes with two things: how anyone knows the data is fresh and correct, and which trade-offs you made on purpose.

Freshness can be as simple as a query that shows the latest date in the model and the time it was loaded, displayed on the dashboard itself. Trade-offs are the decisions where you chose one good thing over another. A daily batch instead of streaming, because the team plans redistribution once a morning. Keeping raw JSON, which costs storage but allows reprocessing. Using a managed scheduler instead of cron, because of dependencies and retries. Naming two or three such choices makes the design sound deliberate rather than accidental.

-- Shown on the dashboard as a small status card
SELECT
    MAX(trip_date)                    AS latest_trip_date,
    MAX(loaded_at)                    AS last_loaded_at,
    COUNT(*) FILTER (
        WHERE trip_date = CURRENT_DATE - 1
    )                                 AS rows_for_yesterday
FROM daily_station_demand;

-- Check used by the check_freshness task
SELECT COUNT(*) = 0 AS is_stale
FROM daily_station_demand
WHERE trip_date = CURRENT_DATE - 1;

"To make freshness visible, the dashboard has a status
card with the latest trip date and load time. If
yesterday has no rows, the DAG's last task fails and
I get an email before the team opens the report.

A few trade-offs I made on purpose: daily batch rather
than streaming, because the team plans once a morning;
keeping raw JSON so I can reprocess; and Airflow over
cron because the weather and trip steps depend on
each other.

If the team wanted hourly updates, the first thing I
would change is the extract, to pull in small windows."

That last sentence, what you would change if requirements changed, is a strong finish. It shows you understand the design is a response to today's needs, not the only possible answer. In Uncodemy mock interviews, candidates who end this way usually get asked a deeper design question, which is exactly the conversation you want.

SECTION 04Build your own pipeline walkthrough

These three tabs turn your pipeline into a structured, five-minute walkthrough. Prepare them once and adjust for each interview.

THE RECORD I WILL FOLLOW: ______________

Born:    where and when is it created? __________
Landed:  where does it first arrive? ___________
         in what form? (raw / cleaned) _________
Shaped:  what changes? __________________________
         where does it end up? __________________
Used:    who uses it, when, for what decision? __

Say all four stages in under 45 seconds.

TASKS (one job each):
  1. ________  2. ________  3. ________  4. ________

DEPENDENCIES:
  ____ must wait for ____ because ____

SCHEDULE: ________  because ____________________

RETRIES: __ times, __ minutes apart
  Why retries are safe here: ____________________

IF EVERYTHING FAILS: who is told, and how? _______

THREE TRADE-OFFS, IN ONE SENTENCE EACH

1. I chose ________ over ________
   because ________________________________

2. I chose ________ over ________
   because ________________________________

3. I chose ________ over ________
   because ________________________________

IF REQUIREMENTS CHANGED TO ________, I WOULD
FIRST CHANGE ____________________________.

SECTION 05Habits that make your pipeline walkthrough shine

These speaking and preparation habits help any pipeline explanation sound clear and senior.

• Draw while you talk — If there is a whiteboard or a shared screen, sketch the four stages as boxes while you narrate the journey. Pictures make dependencies obvious.
• Use the user's clock — Mention when the user needs the data, such as 8 am, and work backwards to the schedule. It shows design driven by real needs.
• Name the raw layer — Explain why you keep an untouched copy of source data. It answers the reprocessing question before it is asked.
• Link retries to safety — Every time you mention a retry, add why it cannot create duplicates. That pairing is a strong signal of experience.
• End with a change plan — Finish with what you would change first if requirements changed. It invites a design conversation on your terms.

SECTION 06Interview Q&A — Pipeline walkthrough

SECTION 07Test yourself — Pipeline walkthrough quiz

SECTION 08Frequently asked questions

SECTION 09Related reads