Data Engineering Path · Airflow
Jinja Templating in Airflow
📝 Making Your DAGs Dynamic with Templates
Airflow uses the Jinja templating engine to inject runtime information into your task parameters. This makes your DAGs dynamic and idempotent — the same DAG can process different dates without any code changes.
Most Used Template Variables
| Variable | Example Output | Description |
|---|---|---|
{{ ds }} |
2024-01-15 |
Logical date (YYYY-MM-DD) |
{{ ds_nodash }} |
20240115 |
Logical date without dashes |
{{ ts }} |
2024-01-15T06:00:00+00:00 |
Full timestamp |
{{ execution_date }} |
datetime object | The execution datetime |
{{ prev_ds }} |
2024-01-14 |
Previous logical date |
{{ next_ds }} |
2024-01-16 |
Next logical date |
{{ dag.dag_id }} |
daily_sales_etl |
DAG identifier |
{{ task.task_id }} |
extract_data |
Current task identifier |
{{ var.value.key }} |
(your variable) | Airflow Variable value |
{{ conn.my_conn.host }} |
db.example.com |
Connection attribute |
Practical Examples
# SQL with date templating
run_query = BigQueryInsertJobOperator(
task_id="aggregate_daily",
configuration={
"query": {
"query": """
SELECT *
FROM `project.raw.events`
WHERE DATE(event_time) = '{{ ds }}'
""",
}
},
)
# S3 path with date partitioning
extract = S3KeySensor(
task_id="wait_for_data",
bucket_key="data/year={{ execution_date.year }}/month={{ execution_date.strftime('%m') }}/day={{ ds_nodash }}/",
)
# Bash with dynamic dates
cleanup = BashOperator(
task_id="cleanup_old_data",
bash_command="hdfs dfs -rm -r /data/staging/{{ ds_nodash }}",
)
⚠️ Important
Jinja templates are only rendered at task execution time, not at DAG parse time. This means you can't use template variables in Python code that runs during DAG parsing (like the DAG definition itself or list comprehensions outside of tasks).
Jinja templates are only rendered at task execution time, not at DAG parse time. This means you can't use template variables in Python code that runs during DAG parsing (like the DAG definition itself or list comprehensions outside of tasks).