Re: [DISCUSS] - LLM-Powered DAG Generation

2025-07-10 Thread Amogh Desai
evin > > > Sent from Outlook for iOS<https://aka.ms/o0ukef> > > From: Amogh Desai > Sent: Tuesday, July 8, 2025 2:04:33 AM > To: dev@airflow.apache.org > Subject: Re: [DISCUSS] - LLM-Powered DAG Generation > > Strongly concur with everything

Re: [DISCUSS] - LLM-Powered DAG Generation

2025-07-10 Thread Kevin Yang
to write proper DAG. Thanks, Kevin Sent from Outlook for iOS<https://aka.ms/o0ukef> From: Amogh Desai Sent: Tuesday, July 8, 2025 2:04:33 AM To: dev@airflow.apache.org Subject: Re: [DISCUSS] - LLM-Powered DAG Generation Strongly concur with everything

Re: [DISCUSS] - LLM-Powered DAG Generation

2025-07-07 Thread Amogh Desai
Strongly concur with everything Jarek had to tell. I would be in favour of a strong NO for this. The use case of having an embedded UI to type in natural language is as good as opening your IDE locally with integrated LLM models and asking it to generate the dag using natural language and drop it

Re: [DISCUSS] - LLM-Powered DAG Generation

2025-07-07 Thread Jens Scheffler
Hi, also sharing the opinion that no dedicated UI is needed for this - but would be very welcoming to share experience and maybe expereince and starting promt to get going. So if you want to post (somewhere, e.g. medium) an article about how this is possible, that migth be worthwile to share.

Re: [DISCUSS] - LLM-Powered DAG Generation

2025-07-07 Thread Kaxil Naik
Yup agreed with Jarek. A strong no from my side. We don't want to allow authoring DAGs from Airflow UI especially just to provide an LLM interface. On Mon, 7 Jul 2025 at 12:27, Jarek Potiuk wrote: > Also you might take a look at Airflow Summit videos > https://www.youtube.com/playlist?list=PLG

Re: [DISCUSS] - LLM-Powered DAG Generation

2025-07-06 Thread Jarek Potiuk
Also you might take a look at Airflow Summit videos https://www.youtube.com/playlist?list=PLGudixcDaxY2NIjMYT8t5zA9KJ47wTCkM -> and look back to 2023. There were at least several talks about using LLMs to generate Airflow Dags, and our users are doing it already - and I guess it's quite natural for

Re: [DISCUSS] - LLM-Powered DAG Generation

2025-07-06 Thread Jarek Potiuk
FYI I added your email directly - because apparently you are not subscribed to devlist - please do subscribe following the "community" tab on our website. I don't want to cut down your wings and excitement, but this is a deliberate choice that Airflow UI does not allow to author DAGs. This is a se

[DISCUSS] - LLM-Powered DAG Generation

2025-07-06 Thread Harikrishnan Girikumar
Hello Team, My name is Harikrishnan(Hari), I have an idea/improvement proposal for Airflow. LLM-powered feature within Apache Airflow to significantly enhance the DAG authoring experience. Users would be able to provide natural language descriptions or queries and leverage Large Language Models (