Yup agreed with Jarek. A strong no from my side. We don't want to allow
authoring DAGs from Airflow UI especially just to provide an LLM interface.



On Mon, 7 Jul 2025 at 12:27, Jarek Potiuk <ja...@potiuk.com> wrote:

> Also you might take a look at Airflow Summit videos
> https://www.youtube.com/playlist?list=PLGudixcDaxY2NIjMYT8t5zA9KJ47wTCkM
> ->
> and look back to 2023. There were at least several talks about using LLMs
> to generate Airflow Dags, and our users are doing it already - and I guess
> it's quite natural for people to generate the Dags with the help of LLMs
> already.
>
> On Mon, Jul 7, 2025 at 8:52 AM Jarek Potiuk <ja...@potiuk.com> wrote:
>
> > FYI I added your email directly - because apparently you are not
> > subscribed to devlist - please do subscribe following the "community" tab
> > on our website.
> >
> > I don't want to cut down your wings and excitement, but this is a
> > deliberate choice that Airflow UI does not allow to author DAGs. This is
> a
> > security feature. And our security model
> >
> https://airflow.apache.org/docs/apache-airflow/stable/security/security_model.html
> > is very clear that "UI users" do not have (and should not have)
> > capabilities of authoring DAGs (not as Python code - that allows
> > arbitrary code execution). Maybe (and that is something we might consider
> > in the future) if there is a declarative way of creating DAGs which does
> > not allow to provide arbitrary code, we could allow that, but we have not
> > even settled on the idea of having a single declarative way of creating
> > Dags.
> >
> > Also Airflow DAGS are just Python Code placed in a folder. And there is
> > absolutely nothing stopping you to open your IDE with Claude , Cursor,
> > Copilot, use the prompt of your choice and ... generate DAGs with LLM.
> > There is absolutely no need to have a UI for that.- all the IDEs out
> there
> > already have a fantastic LLM integration, with capability of adding
> prompt,
> > using MCP servers (there are even several MCP servers for Airflow created
> > by the community and we are discussing about creating our own MCP server
> > https://lists.apache.org/thread/xgd66v6s7zf0xkvy3c7ysqvn4csgmw0 - those
> > IDEs have code completion, syntax check, allow you to interact with the
> > Agents and approve/reject proposals when you are using agents to create
> > your DAGs. They even allow you to use your own models that can be
> RAG-ified
> > based - for example - on the private DAGs your company might have. This
> all
> > works **today**.
> >
> > I don't think personally there is any benefit of creating a similar
> > feature in Airflow UI. I can't see any to be honest. Maybe others have a
> > different opinion or maybe you can explain what benefits you see by
> adding
> > such a "UI feature" to Airflow itself (and also the problem about
> security
> > is extremely important and a huge blocker for the whole idea - until this
> > is somewhat addressed the whole idea is basically impossible to be
> accepted
> > by the community.
> >
> > J.
> >
> >
> > On Mon, Jul 7, 2025 at 8:37 AM Harikrishnan Girikumar <
> > harikrishnangiriku...@gmail.com> wrote:
> >
> >> Hello Team,
> >>
> >> My name is Harikrishnan(Hari), I have an idea/improvement proposal for
> >> Airflow.
> >> LLM-powered  feature within Apache Airflow to significantly enhance the
> >> DAG
> >> authoring experience. Users would be able to provide natural language
> >> descriptions or queries and leverage Large Language Models (LLMs) to
> >> automatically generate and modify Airflow DAGs. This aims to democratize
> >> DAG creation, reduce the learning curve for new users, and accelerate
> the
> >> development of complex workflows.  For example: We can have a UI tab in
> >> Airflow where users can add their respective authentication credentials
> >> for
> >> the LLMs they want to use (OpenAI, Claude or their personal model
> serving
> >> link etc.) they can select their AI from drop down and a chat window to
> >> input queries like: Create a DAG to copy my data from S3 to Postgres and
> >> the code generated would be copied to DAG folder. We can restrict the
> >> Prompts to be strictly for DAG generation for initial trial, further
> down
> >> the line a RAG feature could be added where a Vectorized version of
> >> Airflow
> >> documentation is used to improve the accuracy of DAG creation.
> >>
> >> I am really excited about this feature, this would reduce the learning
> >> curve and improve the interaction for new users. Let me know your
> >> thoughts,
> >> looking forward to hearing from the team.
> >>
> >> Regards,
> >> Hari
> >>
> >
>

Reply via email to