Re: [VOTE] Airflow Providers prepared on October 01, 2025

2025-10-01 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested providers (but lagging Amazon subscription so mainly Fab). Tested with Airflow 3.0.6, 3.1.0 and latest main (LocalExecutor and EdgeExecutor) with Integration Test, via a seondary created user.

Re: [ACCELERATED VOTE] Airflow Providers prepared on September 28, 2025

2025-09-28 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested running together via breeze with Edge Executor on 2.11.0, 3.0.6, 3.1.0 and main and looks good. Eror from rc1 was not visible anymore. On 28.09.25 22:52, Hussein Awala wrote: +1 (binding) On

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

2025-09-28 Thread Jens Scheffler
Hi, oh, two large / related discussions emails.The discussio here is already very fragmented, a lot of ideas have been sketched and it is a very long read to get the context and history. And concepts rotated several times. So also here I'd propose to write an AIP with ideas of potential option

Re: [VOTE] Airflow Providers prepared on September 25, 2025

2025-09-25 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested providers (especially Edge) in Breeze with latest main, and 3.0.6 - These look good. But testing with brand new released 3.1.0 fail in all examples being parsed - assume it is related to stan

Re: [ACCELERATED VOTE] Release Airflow 3.1.0. from 3.1.0rc2 & Task SDK 1.1.0 from 1.1.0rc2

2025-09-24 Thread Jens Scheffler
Airflow Core 3.1.0rc2: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Task SDK 1.1.0rc2: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Tested 3.1.0rc2 release with last release of EdgeExecutor and integration test Dag. Also started

Re: [VOTE] September 2025 PR of the Month

2025-09-24 Thread Jens Scheffler
We had (again) so many cool features and PRs! Is a hard choince again. As I am a UX fanboy I also appreciate the changed coloring (even though I need to get used to new color of "running" state for tasks). Therefore +1 to #53981 On 24.09.25 21:58, Kaxil Naik wrote: Wow! Great candidates. My

Re: [Discuss] Improving Airflow UI Testing Coverage

2025-09-24 Thread Jens Scheffler
Echoing a full agreement to all statement before. I also would emphasize the test pyramid. As we really have a low coverage we should maybe also have a metric. I would also propose that we focus more on testing coverage == hopefully also improved quality preventing regression in 3.2. One optio

Re: [VOTE] Airflow Providers prepared on September 18, 2025

2025-09-20 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Checked my usual route with the "Integration Test" Dag from Edge Provider examples plus Docker Operator on current main, 3.1.0rc1, 3.0.6 and 2.11.0 with Postgres and EdgeExecutor. All in success On

Re: [Discussion] Make the scheduler's task selection algorithm pluggable

2025-09-13 Thread Jens Scheffler
I see a bit of a risk, as the scheduler code is quite complex... (similar like Jarek) if somebody sees this and plugs in, I assume in most cases this make it worse. Also locks us in a plugin API and removes flexibility if we need to change/refactor something. On the other side I fear also a bi

Re: [VOTE] Airflow Providers prepared on September 05, 2025

2025-09-05 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested the changes of Edge Provider in Airflow 3.0.6, 2.10.5 and current main (which hopefully is clode to be 3.1) with breeze and the integration test DAG and all looks good. (By testing just found

Re: [DISCUSS] Request for translation sponsorship: Simplified Chinese (zh-CN) UI translation

2025-09-02 Thread Jens Scheffler
Hi Fortytwoo, thanks for the translation offer. I assume a Simplified Chinese translation would make much sense. I don't know how large the user base is but potentially large. Thanks for offering translation and the PR I saw as well. Without looking at the content technically this looks OK.

Re: [DISCUSS] AIP-94 Decouple Remote Commands from airflow CLI (to airflowctl)

2025-09-02 Thread Jens Scheffler
Hi Constance. the airflow CLI is still needed for some admin commands which arequire DB access as well is used to start the server components. One example is DB migration, DB Cleaning utils. This can not be a remote command (chicken and egg problem). But all (admin) commands which can be ru

Re: [LAZY CONSENSUS] Re: Add Catalan language to Airflow

2025-09-01 Thread Jens Scheffler
:37 PM Jens Scheffler wrote: Forgot the [LAZY CONSENSUS] to be really formal. If no objection then passed by Sept 1st 14:53 CEST On 28.08.25 20:28, Jens Scheffler wrote: Formally for the process we documented at least one PMC need to agree. I am in as sponsor, Is Jarekd "Cool!" and Amou

Re: [Discussion] Should We avoid using AirflowException direclty?

2025-09-01 Thread Jens Scheffler
Also was a bit focussing on other stuff, late to the party. For deprecation we should consider RUFF rules as well. But on side of code changes I fear a lot of people wil have implementations based on AirflowException. I did not get the idea of backwards compatability. If we change Exception ty

Re: [DISCUSS] Updating acceptence criteria for new translations and translation owners

2025-08-29 Thread Jens Scheffler
Thanks Shahar to have the discussion here. I don't see issues with the current model we have. I'd propose to change the rules if we see _real_ issues. I am sure that one or the other language will potentially lose interest and we might need to remove it. But others will be added. I see this as

[LAZY CONSENSUS] Re: Add Catalan language to Airflow

2025-08-28 Thread Jens Scheffler
Forgot the [LAZY CONSENSUS] to be really formal. If no objection then passed by Sept 1st 14:53 CEST On 28.08.25 20:28, Jens Scheffler wrote: Formally for the process we documented at least one PMC need to agree. I am in as sponsor, Is Jarekd "Cool!" and Amough's "Nice!"

Re: Add Catalan language to Airflow

2025-08-28 Thread Jens Scheffler
Formally for the process we documented at least one PMC need to agree. I am in as sponsor, Is Jarekd "Cool!" and Amough's "Nice!" to be interpreted as a +1 from a PMC? Green light? :-D On 28.08.25 17:48, Buğra Öztürk wrote: Great news! Looks cool! On Thu, Aug 28, 2025 at 12:44 PM Amogh Desai

Re: [VOTE] Release Airflow 3.0.6 from 3.0.6rc2 & Task SDK 1.0.6 from 1.0.6rc2

2025-08-26 Thread Jens Scheffler
Hi Kaxil, Keeping me busy with release tests every second day makes me think I need to further automate my release tests :-D Checked Airflow-Core 3.0.6RC2: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Checked Task-SDK 1.0.6RC2: +1 (binding) - Checked SVN, Rep

Re: [VOTE] August 2025 PR of the Month

2025-08-25 Thread Jens Scheffler
I have a hard time as also #54252 was missing and was a longer path to merge it. This is personally the same rank like #51667 and #53035 for me. As I really have a hard choice... I look for a little helper this time: $ python Python 3.12.3 (main, Aug 14 2025, 17:47:21) [GCC 13.3.0] on linux Typ

Re: [VOTE] Release Airflow 3.0.6 from 3.0.6rc1 & Task SDK 1.0.6 from 1.0.6rc1

2025-08-22 Thread Jens Scheffler
Airflow core 3.0.6rc1: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Task SDK 1.0.6rc1: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Like last time: Used breeze and the *.0.6rc1 with current EdgeExecutor and started the "integrat

Re: [DISCUSS] Replace constraints with uv.lock mechanisms for dev env freeze

2025-08-19 Thread Jens Scheffler
I am +1 on the idea and also see this as next logical step. Some thoughts from my side: (1) If we plan to split the distributions to separate api-server, scheduler deployments in future - will we then also as next increment the split into multiple logical uv.locks? Or will it be (for the time

Re: [ACCELERATED VOTE] Release Airflow 3.0.5 from 3.0.5rc3 & Task SDK 1.0.5 from 1.0.5rc3

2025-08-18 Thread Jens Scheffler
Airflow Core 3.0.5rc3: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Task SDK 1.0.5rc3: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Like last time: Used breeze,and the *.0.5rc3 with current EdgeExecutor and started the integrati

Re: [ANNOUNCEMENT] Switched pre-commit to prek in our CI/Breeze

2025-08-17 Thread Jens Scheffler
Very cool! Thanks *Jo* for the support and great tool and Jarek to take the lead! On 17.08.25 10:25, Aritra Basu wrote: This is a good change, looking forward to seeing it adopted more -- Regards, Aritra Basu On Sun, 17 Aug 2025, 12:36 pm Jarek Potiuk, wrote: Hello everyone, *TL;DR: Please

Re: [VOTE] Release Airflow 3.0.5 from 3.0.5rc2 & Task SDK 1.0.5 from 1.0.5rc2

2025-08-14 Thread Jens Scheffler
Airflow core 3.0.5rc2: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Task SDK 1.0.5rc2: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures I tested 3.0.5rc2 via breeze using EdgeExecutor (built from main) with the Integration Test Dag

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-12 Thread Jens Scheffler
Very very cool! First error from the "beta test deck": ``` (airflow) jscheffl@hp860g9:~/Workspace/airflow$ git commit -m "Prevent problems with weaviate-client==4.16.7" Installing rst-backticks ⠙ Installing hooks... error: Failed to install hook `rst-backticks`   caused by: command `pip insta

Re: [VOTE] Airflow Providers prepared on August 12, 2025

2025-08-12 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested standard+fab via breeze in Airflow 3.0.4 and integration test Dag, created a normal user. All looks good after a very simplistic test. On 12.08.25 12:19, Elad Kalif wrote: Hey all, I have ju

Re: [VOTE] Airflow Providers prepared on August 07, 2025

2025-08-09 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Deployed providers via breeze into Airflow 3.0.4 and 2.11.0 (only 2.x compatible providers) and tested the Integration Test with EdgeExecutor and all seems to work well. Jens On 09.08.25 08:14, Amo

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

2025-08-06 Thread Jens Scheffler
Oh, repeating on the "right" thread: Airflow Core 3.0.4rc2: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Note: Source Tar.GZ is building with a differecen but same like RC1 viewing the diff in `diffoscope` shows just a symlink packaged permission diff because

Re: [VOTE] Release Apache Airflow from 3.0.4rc1 and TaskSDK from 1.0.4rc1

2025-08-06 Thread Jens Scheffler
Airflow Core: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Note: Source Tar.GZ is building with a differecen but same like RC1 viewing the diff in `diffoscope` shows just a symlink packaged permission diff because I tested on Linux vs. it was built on MacOS. So

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

2025-08-06 Thread Jens Scheffler
after `Here is a simple test case that makes the benefits of the improvements noticeable` because, it seemed rather long winded detail about a test case. A higher level summary might be helpful to your audience. Is there a PR with your optimization. You wrote "there is a patch" but did not, u

Re: [VOTE] Release Airflow 3.0.4 from 3.0.4rc1 & Task SDK 1.0.4 from 1.0.4rc1

2025-08-05 Thread Jens Scheffler
ng good! Jens, I do not see a difference in the source tarball being different when I test it. We also perform raw byte comparison between tarballs, so I am not so sure why you see that error. Thanks & Regards, Amogh Desai On Tue, Aug 5, 2025 at 3:22 AM Jens Scheffler wrote: Hi Ash!

Re: [VOTE] Release Airflow 3.0.4 from 3.0.4rc1 & Task SDK 1.0.4 from 1.0.4rc1

2025-08-04 Thread Jens Scheffler
Hi Ash! Tested 3.0.4RC1 with latest EdgeExecutor and Integration Test DAG and all looks good! Airflow-Core 3.0.4rc1: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures I am not sure why the source tarball is binary different when I build locally but inspecting the

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

2025-08-03 Thread Jens Scheffler
Hi, We also have a current demand to have a workflow to execute 10k to 100k tasks. Together with @AutomationDev85 we are working on a local solution because we also saw problems in the Scheduler that are not linearly scaling. And for sure not easy to be fixed. But from our investigation also

Re: [VOTE] Airflow Providers prepared on July 29, 2025

2025-07-29 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested Edge3 Package 1.1.2rc1 with Airflow 2.11.0 and 3.0.3 and all seems to be good after execution of the integration_test Dag. On 29.07.25 12:57, Elad Kalif wrote: Hey all, I have just cut the n

Re: [VOTE] July 2025 PR of the Month

2025-07-28 Thread Jens Scheffler
A lot of options this one and hard to decide! The top three listed are the hardest competitors in my view - giving my +1 to #52996 especially as this is from a newcomer as contributor and we had a couple of review cycles and discussions but finally with all reviewers agreed to merge it! On 28.

Re: [DISCUSS] Removing postgres frrom the helm chart (bitnami) ?

2025-07-27 Thread Jens Scheffler
+1 from my side. Like the idea and also this does not people lead to think they can deploy via helm chart and run productive postgres. On 27.07.25 13:51, Aritra Basu wrote: +1 to everything, I think this is a good change to make. I think we should preemptively go ahead with dropping it before w

Re: [DISCUSS] Improve test package hierarchy?

2025-07-27 Thread Jens Scheffler
Hi Jarek, I like the general idea of the structure as you propose. I would not "swap" as you described afterwards but propose to use the first proposal you made. For docker and k8s tests I think it would be better to keep them outside of "airflow-core" as they have a rather integrative/syste

Re: [VOTE] Airflow Providers prepared on July 17, 2025

2025-07-17 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested the standard provider with Airflow 2.11 and 3.0.3 by running the integration_test Dag via EdgeExecutor, all looks good! On 17.07.25 21:23, Elad Kalif wrote: Hey all, I have just cut the new

Re: Declarative DAGs with Pydantic / Hydra

2025-07-16 Thread Jens Scheffler
Hi Tim, thanks for dropping the idea and the context. I think it is fine to post ideas and solutions to the devlist. While I agree it might not be a direct candidate to embed this into the core I think such ideas very good contribute to the wider ecosystem and there are very probably a coupl

Re: Python 3.13 is coming

2025-07-15 Thread Jens Scheffler
Hi Jarek, cool! Thanks for all the efforts and the breath in preparing this! Added some comments in PR gut greatly appreciate (1) that notes describe the upper bindings and (2) that the reference image default stays with 3.12 as 3.13 lags critical providers being ready. Jens On 15.07.25 19:

Re: [VOTE] Release Apache Airflow Helm Chart 1.18.0 based on 1.18.0rc2

2025-07-13 Thread Jens Scheffler
+1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures On 11.07.25 15:29, Jarek Potiuk wrote: +1 (binding). Tested: * reproducibility * signatures * checksums * licences I installed Airflow on local kubernetes using the rc2 helm chart and it worked flawlessly - including

Re: [VOTE] Release Airflow 3.0.3 from 3.0.3rc6 & Task SDK 1.0.3 from 1.0.3rc6

2025-07-12 Thread Jens Scheffler
Airflow-Core: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Task-SDK: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures I was deploying and testing with EdgeExecutor and can say it works and the commonintegration tests are good. Cross

Re: [VOTE] Release Airflow 3.0.3 from 3.0.3rc5 & Task SDK 1.0.3 from 1.0.3rc5

2025-07-10 Thread Jens Scheffler
Airflow-core: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Task-SDK: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Briefly tested Edge Executor in the release and ran the integration test Dag, looks all good. On 10.07.25 22:10,

Re: S3 Dag Bundle Versions and DB Manager

2025-07-09 Thread Jens Scheffler
My 2ct on the discussions are similar like the opinions before. From my Edge3 experience migrating DB from provider - even if technically enabled - is a bit of a pain. Adding a lot of boilerplate, you need to consider your provider should also still be compatible with AF2 (I assume) and once a

Re: [VOTE] AIP-67 Mutlti-team deployment of Airflow Components (RELOADED)

2025-07-08 Thread Jens Scheffler
Thanks Jarek for the rework. I see added complexity for Multi-Team but given the current state of 3.0 I think it is well thought to strip-off complexity. Much better than the original AIP. I like the current approach much more like the previous. +1 binding. Nit: Multi-Team Pluing support migh

Re: [VOTE] Airflow Providers prepared on July 08, 2025

2025-07-08 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Unable to test amazon provider, so this is just the technical release check. On 08.07.25 12:20, Elad Kalif wrote: Hey all, I have just cut amazon ad-hoc Airflow Provider package. This email is callin

Re: Code sharing between Airflow Core and Task SDK - how do we achieve it

2025-07-07 Thread Jens Scheffler
e should find some better logical name for core_and_sdk :) pon., 7 lip 2025, 21:44 użytkownik Jens Scheffler napisał: Cool! Especially the "shared" folder with the ability to have N-combinations w/o exploding project repo root! On 07.07.25 14:43, Ash Berlin-Taylor wrote: Oh, and al

Re: [DISCUSS] Fixing (removing) initializations in "airflow.__init__`) in 3.1 or 3.2 ?

2025-07-07 Thread Jens Scheffler
Thanks Jarek for starting the discussion which I needed to think about (and was busy all day) until reading the thread - is already well evolved with details! I assume also we need to make 1-2 PoCs considering how this is made, more with @decorators (like @requires[ORM] to signal what dependen

Re: Code sharing between Airflow Core and Task SDK - how do we achieve it

2025-07-07 Thread Jens Scheffler
st that has all it needs in itself. Option 1 could very quickly get out of hand and if we decide to separate triggerer / dag processor / config etc etc as separate packages, back compat is going to be a nightmare and will bite us harder than we anticipate. Thanks & Regards, Amogh Desai On

Re: [DISCUSS] - LLM-Powered DAG Generation

2025-07-07 Thread Jens Scheffler
Hi, also sharing the opinion that no dedicated UI is needed for this - but would be very welcoming to share experience and maybe expereince and starting promt to get going. So if you want to post (somewhere, e.g. medium) an article about how this is possible, that migth be worthwile to share.

Re: [VOTE] Airflow Providers prepared on July 03, 2025

2025-07-05 Thread Jens Scheffler
..@gmail.com> wrote: +1 non-binding. Tested with some sample dags LGTM. Pavan. On Thu, Jul 3, 2025 at 9:27 PM Jens Scheffler +1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested Docker, Standard and Edge Provider Packages with EdgeExecutor in

Re: [VOTE] Release Airflow 3.0.3 from 3.0.3rc3 & Task SDK 1.0.3 from 1.0.3rc3

2025-07-05 Thread Jens Scheffler
airflow-core: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Interestingly it showed the file apache-airflow-3.0.3-source.tar.gz be binary different but when un-packing the content is the same. I assume order or metadata of compressed content is different but if e

Re: [VOTE] Airflow Providers prepared on July 03, 2025

2025-07-03 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested Docker, Standard and Edge Provider Packages with EdgeExecutor in Airflow 2.11.0 and 3.0.2 - looks all good. (At least what I tested) On 03.07.25 09:46, Elad Kalif wrote: Hey all, I have just

Re: Code sharing between Airflow Core and Task SDK - how do we achieve it

2025-07-02 Thread Jens Scheffler
I'd also rather prefer option 2 - reason here is it is rather pragmatic and we no not need to cut another package and have less package counts and dependencies. I remember some time ago I was checking (together with Jarek, I am not sure anymore...) if the usage of symlinks would be possible. T

Re: [LAZY,CONSENSUS] Example dags

2025-06-28 Thread Jens Scheffler
tiuk wrote: No need to +1 but ... yeah.. Supporting it by all means. On Sun, Jun 22, 2025 at 11:22 PM Jens Scheffler Hi, as the DICSUSS emails silcenced and also the number of ideas/comments flattened I wanted to call for a LAZY,CONSENSUS for the example dag cleanup/refurbish. Together with A

Re: [VOTE] June 2025 PR of the Month

2025-06-27 Thread Jens Scheffler
+1 for 51735 On 27.06.25 14:56, consta...@astronomer.io.INVALID wrote: Hard choice but +1 for 51153. Docs improvements are very needed. Massive shoutout to all the internationalization work!! Wavered between the two as that is also very needed On Jun 27, 2025, at 8:37 AM, Rahul Vats wrote:

Re: [HELP NEEDED][DISCUSSION] SQLAlchemy v2 support

2025-06-26 Thread Jens Scheffler
Hi, thanks for taking the lead on this! Regarding (1) I fear we need to make a dual version support for a moment because some providers also use SQLA (fab, edge3 at least I know of, standard also soonish if human operators are implemented). If we directyl move to 2.0 then this would be sort o

Re: [DISCUSS] What should we cherry-pick ?

2025-06-24 Thread Jens Scheffler
e backported version). +0.5 for 4. I hope that changes not related to new features will be backported when feasible; however, we can skip them if the required effort is substantial. This is because failing to backport these items could potentially lead to future conflicts with points 1 or 3. Best, W

Re: [DISCUSS] What should we cherry-pick ?

2025-06-23 Thread Jens Scheffler
I am +1 for (1) to (3) [also assuming that (2) is mostly like a doc bugfix!] For (4) I am hesitant and would rather be conservative. Every cherry-pick has a risk to break something in old codebase. As branches change over time and backport PRs are clearly less cautious reviewed it might lead t

Re: [Discussion] AIP-90: Should It Be Implemented in the Standard Provider or as a standalone provider?

2025-06-23 Thread Jens Scheffler
Hi, Thanks Wei for taking the lead in starting to implement! Hope I can review the next days. as I was writing the AIP together with Vikram I was and still am for (=+1) to keep it "human" centric. Also adding an API such that somebody else is able to roll their whatever UI and not being lock

Re: [VOTE] Example dags

2025-06-23 Thread Jens Scheffler
in a '[DISCUSS]` thread might be misleadin - one of the reason to have `[PREFIX]' is to make them aware that there is a vote or consensus going on, where they might have to take look - where '[DISCUSS]' is not really binding for decision. On Sun, Jun 22, 2025 at 10:50 PM Jens Sc

[VOTE] Example dags

2025-06-22 Thread Jens Scheffler
(binding) Jens On 11.05.25 16:24, Jens Scheffler wrote: Hi, following-up on the discussion we had and the verbal discussion in Airflow 3 Dev call May 22nd (see notes in https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3+Dev+call%3A+Meeting+Notes#Airflow3Devcall:MeetingNotes-22May

[LAZY,CONSENSUS] Example dags

2025-06-22 Thread Jens Scheffler
11.05.25 16:24, Jens Scheffler wrote: Hi, following-up on the discussion we had and the verbal discussion in Airflow 3 Dev call May 22nd (see notes in https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3+Dev+call%3A+Meeting+Notes#Airflow3Devcall:MeetingNotes-22May2025) I propose the

Re: [VOTE] Release Apache Airflow Helm Chart 1.17.0 based on 1.17.0rc2

2025-06-22 Thread Jens Scheffler
+1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures On 21.06.25 12:08, Shahar Epstein wrote: +1 non-binding On Wed, Jun 18, 2025 at 10:36 PM Jed Cunningham wrote: Hello Apache Airflow Community, This is a call for the vote to release Helm Chart version 1.17.0. The

Re: Discuss: AIP-67 (multi team) now that AIP-82 (External event driven dags) exists

2025-06-21 Thread Jens Scheffler
Thanks for the rework/update of the AIP-72! Just a few small comments but overall I like it as it is much leaner than originally planned and is in a level of complexity that it really seems to be a benefit to close the gap as described. On 21.06.25 14:52, Jarek Potiuk wrote: I updated the AI

Re: [VOTE] Airflow Providers prepared on June 20, 2025

2025-06-20 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested and can confirm that with Fab it is possible w/o workarounds to create users via UI again. On 20.06.25 16:32, Elad Kalif wrote: Hey all, I have just cut the new wave Airflow Providers packag

Re: Discuss: AIP-67 (multi team) now that AIP-82 (External event driven dags) exists

2025-06-15 Thread Jens Scheffler
Hi all, took a long time digesting all the discussion thread. I think it would be good to rewrite details to a new AIP so that it can be compared with the old AIP. I think this also could include the extension (or is this planned otherwise?) to link multiple Airflow instances via Pub/Sub suc

Re: [VOTE] Airflow Providers prepared on June 15, 2025

2025-06-15 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Tested with Airflow 3.0.2 and 2.11.0 and EdgeExecutor to run some workload all looks good. Bugs seems to be fixed. Found one problem adding a user in UI with Fab, reported details on the ticket http

Re: [VOTE] Adding @RoyLee1224 @guan404ming as Non-Committer Translation Owners and @jason810496 as both a Code Owner and Translation Owner

2025-06-14 Thread Jens Scheffler
+1 binding. I see a lot of activity and would support welcoming all! Thanks for the initiative! On 14.06.25 15:28, Wei Lee wrote: Hi fellow Airflowers, Following the newly passed Internationalization (i18n) Policy [1], I'd like to suggest and invite - @RoyLee1224 - @guan404ming as Non-Commi

Re: [LAZY CONSENSUS] Adding @TJaniF @m1racoli as Non-Committer Translation Owners

2025-06-13 Thread Jens Scheffler
Lazy consensus reached. Merging. Welcome @TJaniF + @m1racoli ! On 10.06.25 23:58, Jens Scheffler wrote: Hi Devs, alongside and after VOTE of the new i18 policy and aiming to follow the new policy as beta tester I'd like to suggest and invite - @TJaniF - @m1racoli ... as additional G

[LAZY CONSENSUS] Adding @TJaniF @m1racoli as Non-Committer Translation Owners

2025-06-10 Thread Jens Scheffler
Hi Devs, alongside and after VOTE of the new i18 policy and aiming to follow the new policy as beta tester I'd like to suggest and invite - @TJaniF - @m1racoli ... as additional German Native speakers (I account swiss as being almost exactly like German :-D ). Both are working professionall

Re: [VOTE] Release Apache Airflow Python Client 3.0.2 from 3.0.2rc1

2025-06-10 Thread Jens Scheffler
+1 (binding) - Checked SVN, Checksums, Reproducible package build, Licenses, Signatures On 10.06.25 22:27, Kaxil Naik wrote: Hey fellow Airflowers, I have cut the first release candidate for the Apache Airflow Python Client 3.0.2. This email is calling for a vote on the release, which will las

Re: [VOTE] Release Airflow 3.0.2 from 3.0.2rc2 & Task SDK 1.0.2 from 1.0.2rc2

2025-06-07 Thread Jens Scheffler
+1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures I attempted to test locally via breeze and generated a strange error when attempting to use EdgeExecutor. Dropped a discussion in https://apache-airflow.slack.com/archives/C03G9H97MM2/p1749315982085709 to qualify if t

Re: [VOTE] Airflow Providers prepared on June 03, 2025

2025-06-04 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures On 03.06.25 19:45, Jarek Potiuk wrote: BTW. I have new key added so PMC members might need to pull the KEYS from SVN to verify signatures On Tue, Jun 3, 2025 at 7:41 PM Jarek Potiuk wrote: Hey all

Re: [VOTE] Internationalization (i18n) policy

2025-06-04 Thread Jens Scheffler
Airflow, as well as related changes in the version release policy. 2. Retrospectively approving the following as both code owners and translation owners in the respective languages, according to the definitions in the policy: - Jens Scheffler - German (de) - Bas Harenslak - Dutch (nl) - Wei

Re: [VOTE] Release Airflow 3.0.2 from 3.0.2rc1 & Task SDK 1.0.2 from 1.0.2rc1

2025-06-03 Thread Jens Scheffler
Oh was sending tooo fast. During check of Task SDK 1.0.2rc1 I am missing the source tarball in the SVN - was this missed uploading? On 04.06.25 00:01, Jens Scheffler wrote: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures On 03.06.25 22:59, Kaxil Naik wrote: Hey

Re: [VOTE] Release Airflow 3.0.2 from 3.0.2rc1 & Task SDK 1.0.2 from 1.0.2rc1

2025-06-03 Thread Jens Scheffler
+1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures On 03.06.25 22:59, Kaxil Naik wrote: Hey fellow Airflowers, The release candidates for *Apache Airflow 3.0.2rc1* and *Task SDK 1.0.2rc1* are now available for testing! This email is calling for a vote on the release,

Re: [DISCUSS] i18n maintanability policy

2025-06-01 Thread Jens Scheffler
Hi all, thanks @shahar for posting the discussion and the document. First hand also added a few comments but in general 99% agreeing to the definitions written there. After the text discussion is settled in a couple of days I'd propose to make it a README in the translations folder (airflow

Re: [VOTE] May 2025 PR of the Month

2025-05-29 Thread Jens Scheffler
+1 for 50626 - will be a massive effort to complete i18n in UI but this was the bootstrap! On 29.05.25 18:14, Briana Okyere wrote: Hey All, It’s once again time to vote for the PR of the Month! With the help of the `get_important_pr_candidates` script in dev/stats, we've identified the follow

Re: [DISCUSS] Ray provider

2025-05-27 Thread Jens Scheffler
Hi, Thanks for the proposal. I assume you have read the https://github.com/apache/airflow/blob/main/PROVIDERS.rst#accepting-new-community-providers docs? By accident I had also a (not maturing) discussion about integration of Ray as cluster backend into Airflow workflows. But I am not sure h

Re: [Lazy Concensus] Naming tweak in Deadlines table

2025-05-26 Thread Jens Scheffler
Hi Denis, from point of content of the name matching to the meaning I agree - but I have doubts because the column name conflicts with the SQL type that has the same same. As we use ORM this is mostly fine but other logic running SQL on the DB might get into conflict if column name "time" is

Re: [VOTE] Release Apache Airflow Python Client 3.0.0 from 3.0.0rc3

2025-05-24 Thread Jens Scheffler
+1 (binding) - Checked SVN, Checksums, Reproducible package build, Licenses, Signatures Also found the EOL diffrence. But this is non-blocking Jens On 23.05.25 15:35, Jarek Potiuk wrote: +1 binding. Checked signatures, checksums, licences are ok. Reproducibility (barring still unsolved EOL in

[DISCUSS] Special Terms in UI Translations?

2025-05-22 Thread Jens Scheffler
Hi Airflow-Devs! As I jumped on the stream to translate the UI and wanted to contribute German in https://github.com/apache/airflow/pull/50929 which is a next step after the enablement in https://github.com/apache/airflow/pull/50626 we came in the review into some UI / UX questions where we a

Re: [DISCUSS] Human in the loop

2025-05-20 Thread Jens Scheffler
Hi All, oh I am a bit "envy" on receiving the email because I am thinking about EXACTLY the same propsal but did not file an AIP to discuss about this... as a matter of personal capacity and I wanted to finish off existing obligations before dropping new ideas. The very same idea for me was

Re: [ANNOUNCE] Apache Airflow 2.11.0 Released

2025-05-20 Thread Jens Scheffler
Very cool! Hope this helps uns as an increment to get to 3.0 soon! On 20.05.25 12:57, Amogh Desai wrote: Thanks & Regards, Amogh Desai On Tue, May 20, 2025 at 2:40 PM Kaxil Naik wrote: Dear Airflow community, I'm happy to announce that Airflow 2.11.0 was just released. The released

Re: [VOTE] Airflow Providers prepared on May 20, 2025

2025-05-20 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures On 20.05.25 12:57, Amogh Desai wrote: +1 non binding. Checked the changes, they look fine. Thanks & Regards, Amogh Desai On Tue, May 20, 2025 at 3:47 PM Kaxil Naik wrote: +1 binding : checked si

Re: [VOTE] Release Airflow 2.11.0 from 2.11.0rc1

2025-05-18 Thread Jens Scheffler
Hi Wei, mhm, checked the PR and somehow this is the wrong target branch. Migration from Airflow 2.2 were cut on main. If you want to fix something it would need to get to v2-11-test. Otherwise I kind of dis-agree on raising a -1 for a migration problem caused by a migration rule in 2.2.0 tha

Re: [VOTE] Release Airflow 2.11.0 from 2.11.0rc1

2025-05-16 Thread Jens Scheffler
+1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Tested 2.11.0rc1 also in our Bosch deployment and did a small load test as regression. All looks good! On 15.05.25 21:52, Kaxil Naik wrote: Hey fellow Airflowers, The release candidates for *Apache Airflow 2.11.0rc1

Re: [VOTE] Airflow Providers prepared on May 14, 2025

2025-05-15 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Agree with Jarek's -1 opinion for cncf.kubernetes I explicitly tested the changes in Edge3 with our deployment in Bosch as well as I did a local breeze setup in Airflow 2.10.5 and current main - all

Re: [DISCUSS] Example dags

2025-05-11 Thread Jens Scheffler
Hi, following-up on the discussion we had and the verbal discussion in Airflow 3 Dev call May 22nd (see notes in https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3+Dev+call%3A+Meeting+Notes#Airflow3Devcall:MeetingNotes-22May2025) I propose the following next steps: 1) start a sepa

Re: [VOTE] Release Airflow 3.0.1 from 3.0.1rc1 & Task SDK 1.0.1 from 1.0.1rc1

2025-05-09 Thread Jens Scheffler
+1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures This is the technical check. Did not have the chance to test the package, will attempt to check basics until Monday and report critical problems. Unfortunately had no free time the last days to check. On 09.05.25 17:

Re: [VOTE] Airflow Providers prepared on May 08, 2025

2025-05-09 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures On 09.05.25 17:19, Kaxil Naik wrote: +1 binding On Fri, 9 May 2025 at 20:46, Hussein Awala wrote: +1 binding On Fri, May 9, 2025 at 8:02 AM Amogh Desai wrote: +1 non binding. This time my chan

Re: [DISCUSS] Migrating to SQLAlchemy 2

2025-05-08 Thread Jens Scheffler
ed, May 7, 2025 at 10:01 PM Jens Scheffler wrote: Hi Avi, all, we had a bit of a discussion about this and don't know if this is very controversal... but: Python 3.9 will run out of support in September. How about if we drop support early in 3.1 so that we migrate main to 3.10 and by t

Re: [VOTE] April 2025 PR of the Month

2025-05-08 Thread Jens Scheffler
Thanks for the reminder @Amogh... totoally forgot about this: +1 for #48528! On 08.05.25 08:17, Jarek Potiuk wrote: Yeah I think our script needs some love :) On Thu, May 8, 2025 at 7:57 AM Amogh Desai wrote: I am sure it would have been a big challenge to identify the candidates this time

Re: [DISCUSS] Migrating to SQLAlchemy 2

2025-05-07 Thread Jens Scheffler
Hi Avi, all, we had a bit of a discussion about this and don't know if this is very controversal... but: Python 3.9 will run out of support in September. How about if we drop support early in 3.1 so that we migrate main to 3.10 and by this remove the deadlock? Jens On 07.05.25 16:48, Abhish

Re: [VOTE] Airflow Providers prepared on May 05, 2025

2025-05-05 Thread Jens Scheffler
+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures Had no time to test packages though. On 05.05.25 12:25, Amogh Desai wrote: +1 non binding. My changes this time were only for CI fixes, those look good. And generally my dag suite tests look fine as

Re: [DISCUSS] from airflow.models or from airflow.sdk ?

2025-05-05 Thread Jens Scheffler
I also attempted to migrate and can say that such movement can be confusing. And most users just hitting "Google" will find the "old" examples for the next period of time. Even today sometimes my results in "Google" refer to outdated serversion and not to "/stable" branch of docs. Unfortunate

Re: [DISCUSS] Example dags

2025-04-30 Thread Jens Scheffler
Thanks for opening the discussion. I have since a longer time the idea that we need to clean and re-work the examples. I like how examples are used in some places in docs and that examples. But at least 50% of them do not add real value to be loaded. I think these are too many. I'd propose t

Re: [DISCUSS} Enabled experiment auto-merge feature

2025-04-30 Thread Jens Scheffler
As there was a call for more opinions. Here I am :-D I understand both positions. As I like AutoMerge very much I am not giving up :-D I'd like to keep it. I think there is still an option in between. Maybe need to draft a bit of thoughts but I think we could build something still around the

Re: [ANNOUNCE] Next Airflow 2 release will be 2.11

2025-04-29 Thread Jens Scheffler
Cool! This means also the "backport-to-2-10-test" is removed (or should not be used anymore)? On 29.04.25 20:02, Amogh Desai wrote: Awesome. Thanks for the update Thanks & Regards, Amogh Desai On Tue, 29 Apr 2025 at 11:19 PM, Jarek Potiuk wrote: Cool! On Tue, Apr 29, 2025 at 7:35 PM Ka

  1   2   >