Hey all, I have updated our meeting notes document to summarize the discussion from our dev call on 8th Aug.
Link: https://cwiki.apache.org/confluence/x/8ApeEg#Airflow3Devcall:MeetingNotes-8August2024 To all those who attended, can you please double-check and add if I have missed anything? To all those who didn't join, if you disagree with anything in the Summary, please voice your opinion. I will send a separate email with the agenda for the next meeting on 22nd August, but if someone has an item for the agenda, please reach out to me or feel free to add it to the doc <https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3+Dev+call%3A+Meeting+Notes#Airflow3Devcall:MeetingNotes-(Proposed)Agenda> . Regards, Kaxil ------ Including the Summary here too (might break formatting): *Catch-up on action items from last call* - AIP-79: Remove Flask AppBuilder as Core dependency <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-79%3A+Remove+Flask+AppBuilder+as+Core+dependency> : - Jed does not have a working POC yet. He will provide an update on the next call. - Jens will draft a high-level plan for Decoupling Providers' connections metadata from FAB when he returns from vacation in the next two weeks. - Airflow 2.10rc1 & 3.0 branching check: - The main branch became the Airflow 3 branch on 09 Aug 2024 as planned. - Guidelines around it have been documented in this PR <https://github.com/apache/airflow/pull/41457> and here <https://github.com/apache/airflow/blob/main/contributing-docs/10_working_with_git.rst#airflow-git-branches> . - StatsD vs Open Telemetry: Shubham mentioned that the AWS team has not encountered any issues so far, but the POC is yet to be completed. He will update about it on the next dev call. *Discussions* - Addressing Active/Paused DAG terminology <https://github.com/apache/airflow/issues/14459> for Airflow 3.0: The Consensus was to change "Active" to a better alternative. This GitHub issue <https://github.com/apache/airflow/issues/41519> should now cover it and is open for anyone to grab it. - Impact on Airflow 3 development while new UI is being built (Brent Bovenzi) - There will be two UIs: the existing UI and the new React UI. Brent Bovenzi will write a "UI dev practices" in the Contributing guide for someone wanting to try out a new UI. The guide will also include details like ensuring that all new features go to the new React UI. - Should we split Webserver, SDK, CLI, Scheduler, and Triggerer into separately installable packages for the users, while the dev code is in same mono-repo? (Jarek Potiuk) - The consensus was to wait until there is decent progress on AIP-72 implementation, but there was a general agreement on splitting the packages out while keeping apache-airflow as a meta-package that brings it all together. Unlike providers, the intention is not to have a separate version scheme or release these separately. - Library choice for different Airflow APIs (API used internally by Webserver, AIP-72 and external API) (Jarek Potiuk) - The consensus was to use Fast API and generate spec from the code. This will mean we remove Gunicorn from the Webserver, which will be a breaking change, but most folks agree that it is worth it. - This will be tracked as part of AIP-84 UI REST API <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-84+UI+REST+API>. Buğra Öztürk has also volunteered to help since it affects AIP-81. - AIP-80: Operator Templating - Some of the attendees strongly opposed breaking Airflow 2's behaviour because they feared it would cause migration issues and that migration tooling wouldn't be sufficient. So, for Airflow 3, we will continue with the existing templating behaviour and add the new (and explicit) feature proposed in the AIP-80. After implementation, if TP finds a good 1-command migration for this, he will propose it on the mailing list with a demo to showcase the migration effort (with 100s of DAGs) to convince that we can break this behaviour but until then the default answer here is, for now, the decision is to keep the Airflow 2 templating behavior in Airflow 3.
