#1 India's Top IT Training Institute
×

About Uncodemy

Know more about who we are and what we stand for.

About Us

About Us

Discover our mission to equip individuals with essential IT skills and industry expertise.

Read More →
UnCodeMy Gallery

UnCodeMy Gallery

Explore highlights from our IT training sessions and success stories.

Read More →
Mentors

Mentors

Meet the industry experts who train and guide you through your course.

Read More →
Free Resources

Free Resources

Free tutorials, interview questions, tools and calculators to help you learn.

Explore →
×

Free Resources

Learn, practice and plan your career — bilkul free.

Tutorials

Tutorials

Step-by-step written tutorials to learn any technology at your own pace.

Read More →
Free Counselling

Free Counselling

Book a free video counselling session and pick the right course for you.

Read More →
Online Compiler

Online Compiler

Write, run and test your code online — no setup required.

Read More →

Data Engineering · Beginner Vocabulary

Data Engineering Terminology Every Beginner Should Learn

ETL, data warehouse, data lake, pipeline, orchestration — data engineers use these words every day. Here is what they actually mean, in plain English.

Tracks
Data Engineering · Beginner Terms Interactive
Term
—
What it is called
Meaning
—
What it means
Why It Matters
—
Where you use it
Term → Meaning → Usage → Skill
Click a tab to explore the vocabulary beginners need most.

Home / Tutorials / Data Engineering / Data Engineering Terminology Every Beginner Should Learn

Data Engineering · Beginner Vocabulary

Data Engineering Terminology Every Beginner Should Learn

"The CDC job failed overnight, so the bronze tables are stale, the dbt models skipped, and the sales dashboard is showing yesterday's numbers." A sentence like this in your first week on a data team can sound like a foreign language. Every term in it is simple once you see where it sits in the journey data takes, from an app's database to a report. This page follows one stream of data, food orders from an app, through that journey and explains each term at the point where you would meet it.

Stage 1: Where data comes from

Source system
Any system that creates data: the app's production database (often called OLTP, for online transaction processing), a payment gateway, a CRM, website click logs, spreadsheets from the finance team.
Ingestion
Copying data out of sources into the data platform. Done in batches (every night, every hour) or as a stream (continuously, event by event).
Full load vs incremental load
A full load copies the whole table every time. An incremental load copies only rows that are new or changed since the last run, usually tracked with an updated_at column or an ID. Incremental is faster but needs care to catch edits and deletes.
CDC (change data capture)
Reading the database's own change log to capture every insert, update and delete as it happens. Tools such as Debezium do this. It is the usual way to get near-real-time copies without heavy queries on the production database.
Event streaming and message queues
Systems such as Apache Kafka hold a continuous, ordered log of events ("order placed", "rider assigned"). Producers write events; consumers read them at their own pace. A topic is a named stream; partitions split it so several consumers can work in parallel.

Stage 2: Changing the data into something usable

ETL and ELT
Extract, Transform, Load means cleaning data before loading it into the warehouse. Extract, Load, Transform means loading raw data first and transforming it inside the warehouse with SQL. Cloud warehouses made ELT common, because their compute is cheap to scale.
Data cleaning
Fixing types, trimming spaces, standardising names ("delhi", " Delhi " and "DELHI" become "Delhi"), removing duplicates and test records.
Idempotency
A job is idempotent if running it twice gives the same result as running it once. This matters because jobs fail and get re-run. Inserting rows blindly on every run creates duplicates; an upsert (update if the key exists, insert if not) avoids that.

Here is a small pipeline that shows extract, transform, an idempotent upsert and an accidental re-run, using SQLite from Python's standard library:

import sqlite3

raw_batch = [  # extracted from the app's order API (sample data)
    {"order_id": 101, "city": " delhi ", "amount": "1200", "updated": "2026-11-15"},
    {"order_id": 102, "city": "Pune", "amount": "650", "updated": "2026-11-15"},
    {"order_id": 101, "city": "Delhi", "amount": "1250", "updated": "2026-11-16"},  # same order, edited later
]

def transform(row):
    return (row["order_id"], row["city"].strip().title(), int(row["amount"]), row["updated"])

wh = sqlite3.connect(":memory:")
wh.execute("CREATE TABLE fact_orders (order_id INTEGER PRIMARY KEY, city TEXT, amount INTEGER, updated TEXT)")

def load(rows):
    wh.executemany("""
        INSERT INTO fact_orders VALUES (?, ?, ?, ?)
        ON CONFLICT(order_id) DO UPDATE SET
            city = excluded.city, amount = excluded.amount, updated = excluded.updated
        WHERE excluded.updated > fact_orders.updated
    """, rows)

clean = [transform(r) for r in raw_batch]
load(clean)
load(clean)          # the job is re-run by mistake: an idempotent load changes nothing
print(wh.execute("SELECT * FROM fact_orders ORDER BY order_id").fetchall())

Output (Python 3.12):

[(101, 'Delhi', 1250, '2026-11-16'), (102, 'Pune', 650, '2026-11-15')]

Order 101 appears once, with its latest amount, even though it arrived twice and the job ran twice. The WHERE excluded.updated > ... condition also protects against an older record overwriting a newer one if batches arrive out of order. Real pipelines use the same idea with MERGE statements in warehouses such as Snowflake, BigQuery or Databricks.

Stage 3: Where the data is stored

TermWhat it isTypical use
Data warehouseA database designed for analysis: structured tables, SQL, fast aggregations over many rows (OLAP)Business reporting, dashboards
Data lakeCheap object storage (Amazon S3, Azure Data Lake Storage, Google Cloud Storage) holding files in any formatRaw history, logs, data for machine learning
LakehouseA lake with a table format (Delta Lake, Apache Iceberg, Apache Hudi) that adds transactions, schema and updatesOne platform for both BI and data science
Data martA smaller, subject-focused slice of the warehouseTables just for finance or just for marketing
Row vs columnar storage
Operational databases store a row's values together, which suits reading one order at a time. Analytical formats such as Parquet store each column together, which suits "sum the amount column over ten million rows" and compresses well.
Partitioning
Splitting a large table or folder by a column, usually date, so a query for last week reads only last week's files.
Medallion layers (bronze, silver, gold)
A naming convention for stages: bronze holds raw ingested data, silver holds cleaned and joined data, gold holds business-ready tables. Some teams say raw, staging and marts instead.

Stage 4: Modelling the data for questions

Fact and dimension tables
A fact table records events with numbers (each order and its amount). Dimension tables describe the things involved (customer, restaurant, date, city).
Star schema
One fact table in the centre joined directly to its dimensions. A snowflake schema further splits dimensions into sub-tables. Star schemas are simpler to query and are what BI tools prefer.
Slowly changing dimension (SCD)
How you handle attributes that change: a restaurant moves from Noida to Gurugram. Type 1 overwrites the old value. Type 2 keeps history by adding a new row with valid-from and valid-to dates, so old orders still report under Noida.
Grain
What one row of a table represents: one order, one order line, or one day per restaurant. Mixing grains in a single table is a frequent cause of double-counted totals.

Stage 5: Running and trusting the pipeline

Orchestration and DAG
An orchestrator such as Apache Airflow, Dagster or Prefect runs jobs in the right order on a schedule. The dependencies form a DAG, a directed acyclic graph: "load orders" must finish before "build daily revenue", and nothing loops back.
dbt
A tool for writing warehouse transformations as SQL files, with tests, documentation and dependencies between models. "The dbt models skipped" means downstream SQL did not run because an upstream step failed.
Backfill
Re-running a pipeline for past dates, for example after fixing a bug. Idempotent jobs make backfills safe.
Data quality checks
Automated tests on the data: no null order IDs, amounts never negative, row counts in a sensible range. Failing checks stop bad data from reaching dashboards.
Freshness, SLA and data lineage
Freshness is how recent the data is. An SLA is the promise ("sales tables ready by 8 a.m."). Lineage is the map of which tables feed which, so you know what breaks when a source changes.
Schema drift
When a source quietly changes, such as a renamed column or a new field. Good pipelines detect it rather than failing silently.

Now the opening sentence decodes: the change-data-capture ingestion failed, the raw layer was not updated, the dependent SQL transformations were skipped by the orchestrator, and the dashboard's freshness SLA was missed.

How to make these terms stick

Reading definitions only goes so far. Build one small end-to-end project: pull data from a public API, land the raw JSON as files, load it into PostgreSQL or DuckDB with an idempotent upsert, model it into a star schema, schedule it with Airflow, and add three data quality checks. You will use most of this page in a weekend. For the tools behind each stage, see data engineering tools and resources for beginners; for how warehouse layers fit together, 3-tier data warehouse architecture; and for a longer-term view of the role, the data engineering career path. Martin Kleppmann's Designing Data-Intensive Applications (O'Reilly, 2017) and Ralph Kimball's The Data Warehouse Toolkit are the standard references for the storage and modelling terms above.