Airbyte & Fivetran — Orchestrating Managed ELT Tools
When Extraction Is Already Somebody Else's Problem
Airbyte and Fivetran both handle the "extract from 300 different SaaS APIs and load into a warehouse" problem so nobody has to write and maintain that connector code themselves. Airflow's role shrinks to exactly one question: trigger their sync, and know when it's done before running whatever depends on that data.
Both operators just call a REST API — Fivetran is SaaS-only, and Airbyte's self-hosted option is a multi-container stack heavier than fits this sandbox. Code-only, no live run.
Airbyte
from airflow.providers.airbyte.operators.airbyte import AirbyteTriggerSyncOperator
trigger_salesforce_sync = AirbyteTriggerSyncOperator(
task_id="trigger_salesforce_sync",
airbyte_conn_id="airbyte_default",
connection_id="4c39a6b2-8c1a-4e9b-9f3e-1a2b3c4d5e6f", # the Airbyte "connection" to sync
asynchronous=False, # task waits for the sync to finish before succeeding
)
Every Airbyte "connection" (a source-to-destination sync pair, configured in the Airbyte UI) has its own UUID — that's what connection_id points at. This operator doesn't configure what gets synced; it only triggers a sync that's already been set up.
Fivetran
from airflow.providers.fivetran.operators.fivetran import FivetranOperator
trigger_stripe_sync = FivetranOperator(
task_id="trigger_stripe_sync",
connector_id="speak_reformed", # Fivetran's own connector identifier
fivetran_conn_id="fivetran_default",
)
Conceptually identical to Airbyte's operator — Fivetran also owns the actual connector configuration in its own UI; Airflow's job is purely "start this sync, wait for it."
The Pattern After Triggering
Both tools' syncs are asynchronous by nature — the operator either blocks until completion (asynchronous=False) or returns immediately and needs a sensor to check status separately:
from airflow.providers.airbyte.sensors.airbyte import AirbyteJobSensor
trigger_sync = AirbyteTriggerSyncOperator(
task_id="trigger_sync",
airbyte_conn_id="airbyte_default",
connection_id="4c39a6b2-8c1a-4e9b-9f3e-1a2b3c4d5e6f",
asynchronous=True,
)
wait_for_sync = AirbyteJobSensor(
task_id="wait_for_sync",
airbyte_conn_id="airbyte_default",
airbyte_job_id=trigger_sync.output,
)
trigger_sync >> wait_for_sync
The natural pattern: a task that waits for the sync (as above), then declares
outlets=[raw_data_asset] - so every downstream transform DAG (dbt/Cosmos, a Great Expectations check) can be scheduled purely on "the raw data changed," with no direct knowledge of Airbyte or Fivetran at all.