Azure Data Factory Interview Questions

Azure Data Factory (ADF) is Microsoft's cloud-based data integration service. These questions cover the concepts interviewers most frequently probe, from pipeline design to performance tuning.

What is Azure Data Factory?

Azure Data Factory is a cloud-based ETL and data integration service that lets you create data-driven workflows to move and transform data at scale, without writing custom code for every step.

What are the core components of ADF?

Pipelines (logical groupings of activities), activities (individual processing steps), datasets (representations of data structures), linked services (connection configurations), and triggers (execution schedulers).

What is the difference between a pipeline and an activity?

A pipeline is a container that orchestrates a set of activities to perform a task; an activity is a single processing step within that pipeline, such as copying data or running a stored procedure.

What is an Integration Runtime (IR)?

The Integration Runtime is the compute infrastructure ADF uses to provide data integration capabilities such as data movement, activity dispatch, and SSIS package execution, across networks.

What are the types of Integration Runtime?

Azure IR (for cloud-to-cloud data movement), Self-hosted IR (for connecting to on-premises or private network data sources), and Azure-SSIS IR (for running SSIS packages in the cloud).

What is the Copy Activity used for?

The Copy Activity moves data from a source data store to a sink data store, optionally performing basic transformations like column mapping and format conversion along the way.

What is a Mapping Data Flow?

A Mapping Data Flow is a visually designed data transformation feature in ADF that runs on managed Spark clusters, letting you build transformation logic without writing code.

What is the difference between Mapping Data Flow and Wrangling Data Flow?

Mapping Data Flows are built for code-free ETL transformations at scale using Spark, while Wrangling Data Flows use a Power Query-like interface for exploratory data preparation.

What are triggers in ADF, and what types exist?

Triggers determine when a pipeline runs. ADF supports schedule triggers (time-based), tumbling window triggers (periodic, stateful intervals), and event-based triggers (fired by events like blob creation).

How do you handle parameterization in ADF pipelines?

Pipelines, datasets, and linked services can accept parameters, which lets you reuse a single pipeline definition across multiple sources, destinations, or environments instead of duplicating pipelines.

What is the difference between ADF and SSIS?

SSIS is an on-premises ETL tool tied to SQL Server, while ADF is a cloud-native, serverless data integration service that can also lift-and-shift and run existing SSIS packages via Azure-SSIS IR.

How do you monitor pipeline runs in ADF?

ADF provides a Monitor tab showing pipeline and activity run history, execution duration, and error details, and it can be integrated with Azure Monitor and Log Analytics for alerting.

What is a Lookup activity?

A Lookup activity retrieves a dataset or a single value from a source, which can then be used to drive control flow logic elsewhere in the pipeline, such as in a ForEach or If Condition activity.

How does ADF handle incremental data loading?

Incremental loads typically use watermark columns (like a last-modified timestamp) or change tracking on the source, with the pipeline querying only records that changed since the previous run.