Creating DAGs with an AI Agent

Ask an AI client to write an Airflow DAG, check the code it produces, and publish the DAG to Airflow. Airflow MCP uses two steps, review and then upload, so AI-written code can't reach Airflow until you approve it.

IMPORTANT
Only users with Admin access can create DAGs using Airflow MCP.

About the review-and-upload process

Airflow MCP will not allow an AI client to write Airflow DAG code straight into Airflow. Publishing a DAG uses two tools:

Step Tool What happens Changes Airflow?
  1. Review
 reviewDagForUpload Saves the proposed DAG Python code to a temporary staging file on the Data Access Gateway node. Returns a short-lived reviewToken. No
2. Upload uploadDag Uses the reviewToken to upload the approved file to Airflow's /dags/upload REST endpoint. Yes

You approve the code and the file name between these two steps. uploadDag is labeled destructive because it can overwrite an existing DAG file with the same name.

Before you begin

  • An administrator has turned on Airflow MCP, and your AI client is connected.
  • You have permission to upload DAGs in Airflow.
  • The DAG file is no larger than the upload limit. The default is 10 MB, set by mcp.airflow.dag-upload-max-size-bytes.
  • Your AI client asks for approval before it runs tools that change or delete data. Most clients do this by default for tools labeled destructive.

Creating and publishing an Airflow DAG

  1. Describe the DAG you want.

    Include the schedule, tasks, dependencies, and any connections, variables or pools it should use. For example:

    "Write an Airflow DAG named daily_sales_etl that runs every day at 02:00. It should run a Spark job, then a data quality check, and email the team if the check fails. Use the existing spark_default connection."

  2. Ask the client to stage the DAG for review.

    The client calls reviewDagForUpload with the proposed code and file name. It gets back a reviewToken. At this point, nothing has been sent to Airflow.

  3. Check the code and file name.
    Review the code the client shows you, and check:
    1. File name:make sure it is what you expect and won't overwrite a DAG you want to keep. Use listDags or getDag to see if a DAG with that name already exists.
    2. Imports and operators: watch for BashOperator, PythonOperator or other operators that run shell commands or any Python code.
    3. Connections, variables and pools: make sure they exist and are the correct ones.
    4. Schedule and catchup: check the schedule, the start_date, and whether catchup could start a lot of backfill runs.
    5. run_as_user: check that it's not set, or set to a user who's allowed to start the DAG. If it doesn't match the user who starts the DAG through Airflow MCP, the run is rejected.
    6. Secrets: make sure there are no passwords, tokens or keys written into the code.
  4. Approve or ask for changes.
    • If you need changes, tell the client what to fix. It stages the new version with reviewDagForUpload and gets a new reviewToken. Check the new version before you approve it.
    • If the code is correct, approve the file name and the code.
  5. Upload the DAG.

    The client calls uploadDag with the reviewToken. When your client asks you to approve this tool call, check that the file name matches what you reviewed, then approve it.

  6. Check that Airflow loaded the DAG.
    The Airflow DAG Processor has to read the new file before the DAG shows up, so it can take a short time to appear. Ask the client, for example:

    • "Is daily_sales_etl listed now?" (listDags or getDag)

    • "Are there import errors for daily_sales_etl?" (listImportErrors)

    • "Show me the source of daily_sales_etl." (getDagSource) Compare it with the code you approved.

  7. Optional: run it for the first time

    New DAGs may be paused, depending on your Airflow settings. Ask the client to unpause the DAG (unpauseDag) or start a test run (triggerDag or triggerDagAndWait). If the run fails, ask the client to look into it with diagnoseDagRun.

Example conversation

You The AI client
"Write a DAG that copies yesterday's files from /data/landing to /data/archive every night at 01:00." Writes the DAG code and calls reviewDagForUpload with the file name nightly_archive.py. Shows you the code and the file name.
"Use the default_pool and set retries to 2." Changes the code, calls reviewDagForUpload again, and shows you the new version.
"Looks good. Upload it." Asks you to approve uploadDag, then calls it with the reviewToken.
"Did it load?" Calls listImportErrors and getDag, and confirms the DAG is there and paused.
"Unpause it and run it once now." Calls unpauseDag, then triggerDagAndWait, and reports the result.

What the client actually does depends on the client and the model.