Data Engineering · Glossary 2026
Data Engineering Glossary: Key Words Explained
Quick summary — data engineering glossary
Data engineering has its own vocabulary. Before you can build pipelines or interview for data engineering roles, you need to understand core concepts — ETL/ELT, pipelines, orchestration, warehouses, lakes, dbt, Spark, and Kafka. Here's a plain-English glossary of the words every beginner should know in 2026.
In this guide you will learn:
- ETL vs ELT — the core pipeline concepts.
- Pipelines and orchestration — how data flows.
- Storage layers — warehouse vs lake vs lakehouse.
- Transformation tools — dbt, Spark, and Python.
- Streaming and messaging — Kafka and real-time data.
SECTION 01ETL vs ELT — core pipeline concepts
Two terms dominate every data engineering conversation. Knowing the difference tells you how a data system is architected.
| Term | Meaning | Best For |
|---|---|---|
| ETL | Extract, Transform, Load — transform before loading | Legacy systems, tight governance |
| ELT | Extract, Load, Transform — load first, transform in warehouse | Modern cloud warehouses (Snowflake, BigQuery) |
| Reverse ETL | Push warehouse data back to operational tools | Syncing data to CRM, marketing tools |
| CDC | Change Data Capture — track changes in real time | Keeping replicas in sync, streaming updates |
| Batch processing | Process data in chunks on a schedule | Daily reports, monthly analytics |
# ETL: transform before loading
import pandas as pd
from sqlalchemy import create_engine
# Extract
df = pd.read_csv("raw_sales.csv")
# Transform
df["revenue"] = df["quantity"] * df["price"]
df = df[df["revenue"] > 0]
df["month"] = pd.to_datetime(df["date"]).dt.to_period("M")
# Load
engine = create_engine("postgresql://user:pass@db/analytics")
df.to_sql("sales_clean", engine, if_exists="replace", index=False)
# Transformation happened BEFORE loading.-- ELT: load raw first, transform in warehouse
-- Step 1: Load raw data (handled by Fivetran/Airbyte)
-- Step 2: Transform using dbt
-- models/sales_clean.sql
SELECT
id,
date,
quantity * price AS revenue,
EXTRACT(MONTH FROM date) AS month
FROM {{ source('raw', 'sales') }}
WHERE quantity * price > 0;
-- Transformation happens INSIDE the warehouse.
-- Modern, more flexible, more maintainable.SECTION 02Pipelines and orchestration
A pipeline moves data from sources to destinations. Orchestration ensures the steps run in the right order, at the right time.
| Term | Meaning | Example |
|---|---|---|
| Pipeline | A series of steps that move and transform data | Source → Clean → Warehouse → Dashboard |
| Orchestration | Scheduling and coordinating pipeline steps | Airflow, Dagster, Prefect |
| DAG | Directed Acyclic Graph — the flow of tasks | Airflow's core concept |
| Job | A single unit of work within a pipeline | "Load daily sales" |
| Idempotency | Running the same job twice produces the same result | Safe retries |
| Backfill | Re-processing historical data | Filling in missing days |
SECTION 03Storage — warehouse, lake, lakehouse
Where does all this data actually live? Three main storage patterns come up repeatedly.
| Term | Meaning | Tools |
|---|---|---|
| Data Warehouse | Structured data, optimized for analytics (SQL) | Snowflake, BigQuery, Redshift |
| Data Lake | Any data (structured or unstructured), cheap storage | S3, GCS, ADLS |
| Data Lakehouse | Combines lake flexibility + warehouse performance | Databricks, Delta Lake, Iceberg |
| OLAP | Online Analytical Processing — for analytics | Warehouses are OLAP systems |
| OLTP | Online Transaction Processing — for operations | PostgreSQL, MySQL |
| Parquet | Columnar file format optimized for analytics | Standard format in data lakes |
SECTION 04Transformation — dbt, Spark, Python
Transformation is where raw data becomes usable. These are the tools and terms you'll encounter most.
| Term | Meaning | When to Use |
|---|---|---|
| dbt | SQL-based transformation framework | In-warehouse transformations |
| Spark / PySpark | Distributed processing engine for big data | Large-scale processing |
| Pandas | Python data library for smaller datasets | Local transforms, scripts |
| Data Modeling | Structuring data into fact/dimension tables | Analytics-friendly schemas |
| Star Schema | Fact table surrounded by dimension tables | Simple, fast queries |
| Slowly Changing Dimensions | Tracking changes in dimension data over time | Historical reporting |
SECTION 05Streaming and messaging
Not all data is processed in batches. Streaming handles events as they happen — critical for real-time use cases.
- Kafka — the most popular event streaming platform.
- Producer / Consumer — who writes and who reads from a stream.
- Topic / Partition — Kafka's structure for organizing events.
- Stream processing — processing events as they arrive (Spark Streaming, Flink).
- At-least-once / at-most-once / exactly-once — delivery guarantees.
- Schema registry — managing schemas for streaming data (Avro, Protobuf).
- Event time vs processing time — when the event occurred vs when it was processed.
SECTION 06Interview Q&A — Data Engineering Terms
Q1What's the difference between ETL and ELT?
ETL transforms data before loading it into the warehouse. ELT loads raw data first, then transforms it inside the warehouse. ELT is more common in modern cloud architectures.
Q2What is a DAG in data engineering?
A DAG (Directed Acyclic Graph) is a dependency graph where tasks flow in one direction with no cycles. Airflow uses DAGs to represent pipeline workflows.
Q3What's the difference between a warehouse and a lake?
A warehouse stores structured, query-optimized data (Snowflake, BigQuery). A lake stores any data type cheaply (S3). A lakehouse combines both.
Q4What is idempotency and why does it matter?
Idempotency means running a job twice produces the same result. It matters because pipelines fail and get retried — without idempotency, you'd duplicate data.
Q5What is Kafka used for?
Kafka is an event streaming platform — it captures real-time events from producers and delivers them to consumers. It's used for real-time analytics, event-driven systems, and stream processing.
SECTION 07Test yourself — Data Engineering Glossary
Five questions. No sign-up.
0 / 5Pick an answer to see why it is right or wrong.
SECTION 08Frequently asked questions
What's the difference between ETL and ELT?
ETL transforms data before loading. ELT loads raw data first, then transforms inside the warehouse. ELT is the modern default for cloud data warehouses.
What is a data pipeline?
A pipeline is a series of steps that moves data from source systems to destinations — cleaning, transforming, and loading it along the way.
What is dbt used for?
dbt is a SQL-based transformation framework. It lets you write models (SQL files), test them, document them, and deploy them inside your warehouse.
What is a lakehouse?
A lakehouse combines the flexibility of a data lake (any data type, cheap storage) with the performance of a warehouse (SQL, analytics). Databricks and Delta Lake are popular examples.
Do I need to know Spark as a beginner?
Not immediately. Master SQL and Python first. Spark becomes important when you work with large data volumes (usually 1-2 years into your data engineering career).
SECTION 09Related reads
Classroom & online · Noida
Master data engineering the right way
Our Data Analytics Training Course covers SQL, Python, PySpark, Airflow, dbt, and cloud — with hands-on projects and placement support.
₹24,500 · full programme- SQL, Python, PySpark, Airflow
- dbt, cloud, real-time data
- Hands-on pipeline projects
- Placement support