Re: [DISCUSS] Extendable parsing controls (a.k.a. Pluggable Parsers)

Hello everyone, As Airflow 3.0 is getting to a final shape I've made a major revision of AIP-85 based on Airflow 3.0 implementation and PoC that I've prepared at https://github.com/apache/airflow/pull/45684

Re: [DISCUSS] Extendable parsing controls (a.k.a. Pluggable Parsers)

Small update: I've created the page on wiki: https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-85+Extendable+DAG+parsing+controls. I suggest moving inline-comments there instead of the originally linked doc. On Wed, Jul 31, 2024 at 1:57 AM Kaxil Naik wrote: > Hi Igor, > > I have added perm

Re: [DISCUSS] Extendable parsing controls (a.k.a. Pluggable Parsers)

Hi Igor, I have added permissions to your cWiki username: ikholopov I am interested in this topic but my feedback will be a bit delayed since I am focussing on reviewing 3.0 items first. Regards, Kaxil On Tue, 30 Jul 2024 at 15:09, Igor Kholopov wrote: > Hi all, > > I've prepared a proposal o

[DISCUSS] Extendable parsing controls (a.k.a. Pluggable Parsers)

Hi all, I've prepared a proposal of the AIP aiming to increase the flexibility and structure up the DAG processing/ingestion in Airflow 3.0+. This proposal largely builds on the accepted AIP-66 & AIP-72. It strives to unblock paths for faster development in remote environments, precise control of