Capstone 4: Automated Code & Airflow DAG Review Bot

+---------------------------------------------------------------------------------+
|                           PROJECT 4 ARCHITECTURE                                |
+---------------------------------------------------------------------------------+
|  GitHub Webhook (PR Opened) -> Fetch Diff -> Airflow AST Static Analysis        |
|                                                      |                          |
|                                                      v                          |
|  LLM Reviewer               -> Detects dynamic top-level code, missing catchup  |
|                                                      |                          |
|                                                      v                          |
|  GitHub Pull Request        <- Automated Review Comments on Diff Line Numbers   |
+---------------------------------------------------------------------------------+

Objective

Automatically scan Airflow DAG pull requests to block top anti-patterns:

1. Dynamic top-level database connections or heavy computations outside tasks.
2. Missing catchup=False.
3. Hardcoded secrets or unrotated credentials.
4. Non-idempotent task patterns.

Code Example: Automated DAG Linter

import ast
from pydantic import BaseModel
from openai import OpenAI
import instructor

client = instructor.from_openai(OpenAI())

class DAGReviewResult(BaseModel):
    has_critical_antipattern: bool
    findings: list[str]
    suggested_rewrite: str
    is_approved: bool

def review_airflow_dag_code(dag_code: str) -> DAGReviewResult:
    prompt = f"""
Review the following Apache Airflow DAG Python file against senior engineering best practices:

1. Check for expensive top-level code execution (queries outside operators).
2. Check if catchup is explicitly set.
3. Check for proper retry policies.
4. Check for hardcoded API keys or database passwords.

CODE:
{dag_code}
"""
    return client.chat.completions.create(
        model="gpt-4o-mini",
        response_model=DAGReviewResult,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.1
    )