Theoretical Quiz — Module 1
Answer these questions to validate your understanding of Airflow's core concepts. Try to answer without looking back at the material.
Q1. What does the acronym DAG stand for, and why is the "Acyclic" property important?
Show Answer
Directed Acyclic Graph. The "Acyclic" property is critical because it prevents infinite loops. If task A depends on task B which depends on task A, the pipeline would never complete. The acyclic constraint ensures every workflow has a clear start and end.
Q2. Explain the difference between the Control Plane and Data Plane in the context of Airflow. Give an example.
Show Answer
- Control Plane (Airflow): Orchestrates when and in what order tasks run. Uses minimal compute resources. Example: Airflow scheduler triggers a task at 6 AM.
- Data Plane (External Systems): Actually processes the data. Uses heavy compute. Example: Apache Spark processes 500 GB of data.
Airflow sends a spark-submit command (control plane) → Spark runs the job (data plane).
Q3. Name the 5 core components of Airflow's architecture and describe each in one sentence.
Show Answer
- Scheduler — Monitors DAGs and triggers tasks based on schedules and dependencies
- Webserver — Provides the UI for monitoring DAG runs and task states
- Workers — Execute the tasks dispatched by the scheduler
- Metadata Database — Stores all state, configuration, connections, and history
- Triggerer — Handles deferrable tasks efficiently without blocking workers
Q4. What are 3 key advantages of Airflow over cron jobs?
Show Answer
- Dependency management — Tasks only run when upstream dependencies succeed
- Rich monitoring UI — Grid, Graph, and Gantt views for real-time pipeline visibility
- Automatic retries and alerting — Self-healing pipelines with email/Slack notifications
Q5. What is XCom and when should you use it? When should you NOT use it?
Show Answer
XCom (Cross-communication) allows tasks to exchange small messages (metadata, row counts, file paths).
- ✅ Use for: Passing file paths, row counts, status flags, small config values
- ❌ Don't use for: Passing large datasets (DataFrames, raw files). Use external storage (S3, GCS) instead.
XCom values are stored in the metadata database, so passing large data will cause database bloat and performance issues.
Q6. Explain the difference between Operator, Sensor, and Hook in Airflow.
Show Answer
- Operator: A template for a task that performs an action (e.g.,
PythonOperatorruns Python code,BashOperatorruns shell commands) - Sensor: A special type of operator that waits for a condition to be true (e.g.,
S3KeySensorwaits for a file to appear in S3) - Hook: A low-level interface for connecting to external systems (e.g.,
PostgresHookmanages database connections). Hooks are used inside operators.