Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

2025-08-07 Thread Wei Lee
+1 (non-binding), Tested the latest airflow-core, task-sdk dep resolution and some Dags. Everything works fine. Best, Wei > On Aug 8, 2025, at 2:32 PM, GUAN-MING CHIU wrote: > > +1 (non-binding). Verified by testing UI and running several dags. > > Best, > Guan-Ming > > Ankit Chaurasia 於 20

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

2025-08-07 Thread GUAN-MING CHIU
+1 (non-binding). Verified by testing UI and running several dags. Best, Guan-Ming Ankit Chaurasia 於 2025年8月8日 週五 下午2:28寫道: > +1 (non-binding). Ran dags and tested my changes. > > *Ankit Chaurasia* > > > > > > > On Fri, Aug 8, 2025 at 11:59 AM Rahul Vats wrote: > > > +1 (non-binding). Verified

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

2025-08-07 Thread Ankit Chaurasia
+1 (non-binding). Ran dags and tested my changes. *Ankit Chaurasia* On Fri, Aug 8, 2025 at 11:59 AM Rahul Vats wrote: > +1 (non-binding). Verified running dags with multiple executors. Did some > UI checks. All looks good! > > Regards, > Rahul Vats > > On Thu, 7 Aug 2025 at 17:36, Pierre J

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Jarek Potiuk
Plus if we get to monorepo - we would have to also implement complexity of that in breeze :( On Fri, Aug 8, 2025 at 8:25 AM Jarek Potiuk wrote: > > In terms of installation, are we looking at `uv tool install prefligit` > or are we looking to > do binary installation? > > I think it does not mat

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Jarek Potiuk
> In terms of installation, are we looking at `uv tool install prefligit` or are we looking to do binary installation? I think it does not matter - it just will need to be installed - but in our docs I think we should recommend `uv tool` as we anyhow require uv and then it's easy to manage all ins

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

2025-08-07 Thread Rahul Vats
+1 (non-binding). Verified running dags with multiple executors. Did some UI checks. All looks good! Regards, Rahul Vats On Thu, 7 Aug 2025 at 17:36, Pierre Jeambrun wrote: > +1 (binding) > > On Thu, Aug 7, 2025 at 10:29 AM Jarek Potiuk wrote: > > > All good then, as the Venv operator is in th

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Amogh Desai
I am really excited for this one and kept reading it as "preflight" until pointed out. The fact that it is 10x faster + built in `uv` support + separate pre-commit per directory (upcoming) is really cool! In terms of installation, are we looking at `uv tool install prefligit` or are we looking to

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Aritra Basu
I do think the closeness of the name warrants making it obvious the difference in docs. I had a few moments of confusion myself. -- Regards, Aritra Basu On Fri, 8 Aug 2025, 9:02 am Jarek Potiuk, wrote: > Yes. Initially I thought the same ("odd choice"). > > That's a good point and something that

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Jarek Potiuk
Yes. Initially I thought the same ("odd choice"). That's a good point and something that we will have to all learn :). I even thought that we should maybe leave `breeze static-checks` as wrapper - only because `prefligit` is not something that one would easily use. However - as most of us use auto

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Tzu-ping Chung
I didn’t even realise the name is NOT preflight before you pointed it out, Daniel… TP -- Sent from my iPhone > On 8 Aug 2025, at 07:11, Daniel Standish > wrote: > > I thought `prefligit` was a typo of `preflight` > > bit of an odd choice in name > > but, i guess it's probably not that bad

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Daniel Standish
I thought `prefligit` was a typo of `preflight` bit of an odd choice in name but, i guess it's probably not that bad of a choice to avoid collisions with `preflight` On Thu, Aug 7, 2025 at 12:28 PM Jarek Potiuk wrote: > Indeed! Jo is amazing :) > > On Thu, Aug 7, 2025 at 8:24 PM Damian Shaw >

Re: [DISCUSS] Best practices for initializing `ObjectStoragePath`

2025-08-07 Thread Kevin Yang
That could be some vault services, https://airflow.apache.org/docs/apache-airflow-providers-hashicorp/stable/secrets-backends/hashicorp-vault.html. Probably thinking it as a key-value store, and looking back to that example from this perspective DEV Environment “key”: FILE_PATH “value”: file://

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Jarek Potiuk
Indeed! Jo is amazing :) On Thu, Aug 7, 2025 at 8:24 PM Damian Shaw wrote: > Already fixed and released! > > -Original Message- > From: Damian Shaw > Sent: Thursday, August 7, 2025 12:28 PM > To: dev@airflow.apache.org > Subject: RE: [DISCUSS] Upcoming pre-commit -> prefligit change > >

RE: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Damian Shaw
Already fixed and released! -Original Message- From: Damian Shaw Sent: Thursday, August 7, 2025 12:28 PM To: dev@airflow.apache.org Subject: RE: [DISCUSS] Upcoming pre-commit -> prefligit change FYI I found two small issues trying to use it as a drop-in replacement for my work environme

RE: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Damian Shaw
FYI I found two small issues trying to use it as a drop-in replacement for my work environment: https://github.com/j178/prefligit/issues/387 https://github.com/j178/prefligit/issues/388 But my otherwise quite complicated .pre-commit-config.yaml (which uses anchors and aliases and remote and loc

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Aritra Basu
Definitely agree with both of you, will be trying this out myself as well. Definitely looking forward to seeing alternatives in the space! -- Regards, Aritra Basu On Thu, 7 Aug 2025, 9:24 pm Jarek Potiuk, wrote: > > Pre-commit is great for its stability but is really failing in terms of > innova

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Jarek Potiuk
> Pre-commit is great for its stability but is really failing in terms of innovation, the project itself does not allow any discussion of using new standards. Had my fair share of those discussions in the past and I quite agree. There is huge difference between "stability" and "stagnation/stubborn

RE: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Damian Shaw
I just want to say I am very excited to see innovation in this space! Pre-commit is great for its stability but is really failing in terms of innovation, the project itself does not allow any discussion of using new standards. I will be testing it out in my own environments and then promoting i

Re: [VOTE] Airflow Providers prepared on August 07, 2025

2025-08-07 Thread Jarek Potiuk
+1 (binding) - everything minus standard provider -1 (binding) standard provider Checked: * reproducibility * existing files check * docker installation * licences * signatures * checksums Why (-1) on standard? Technically this is not a regression - because we have released it in 1.5.0 but we h

[DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-07 Thread Jarek Potiuk
Hello everyone, Early warning about upcoming pre-commit/prefligit change.. Together with Ash and creator of the prefligit: https://github.com/j178/prefligit - we are testing and helping to close the gaps between prefligit and pre-commit (and later we hope we will be able to improve our prefli

Re: [DISCUSS] Best practices for initializing `ObjectStoragePath`

2025-08-07 Thread Josef Šimánek
čt 7. 8. 2025 v 15:04 odesílatel Kevin Yang napsal: > > Maybe join this conversation late, but I would like to share/document a > pattern and hope which can provide more references for the discussion. > > In terms of deployment model, there are isolated environments, each with > their own deploy

Re: [DISCUSS] Best practices for initializing `ObjectStoragePath`

2025-08-07 Thread Kevin Yang
Maybe join this conversation late, but I would like to share/document a pattern and hope which can provide more references for the discussion. In terms of deployment model, there are isolated environments, each with their own deployment of Airflow instance and the required services. DEV Environ

Re: [DISCUSS] AIP-92 Isolate DAG parsing logic

2025-08-07 Thread Jarek Potiuk
Ah.. So if we are talking about a more complete approach - seeing those comments from Ash - make me think if we should have another AIP. (connected) about splitting the distributions. We have never finalized it (nor even discussed id) but Ash - you had some initial document for that. So maybe we sh

Re: [DISCUSS] AIP-92 Isolate DAG parsing logic

2025-08-07 Thread Jarek Potiuk
Also > 1. The Authentication token. How will this long lived token work without being insecure. Who and what will generate it? How will we identify top-level requests for Variables in order to be able to add Variable RBAC/ACLs. This is an important enough thing that I think it needs discussion bef

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

2025-08-07 Thread Pierre Jeambrun
+1 (binding) On Thu, Aug 7, 2025 at 10:29 AM Jarek Potiuk wrote: > All good then, as the Venv operator is in the standard provider and not > part of this vote > > Indeed. We could release it separately and manually update constraints at > release time - before preparing images to use the previou

Re: [DISCUSS] AIP-92 Isolate DAG parsing logic

2025-08-07 Thread Ash Berlin-Taylor
This AIP is definitely heading in the right direction and is a feature I’d like to see. For me the outstanding things that need more detail: 1. The Authentication token. How will this long lived token work without being insecure. Who and what will generate it? How will we identify top-level re

Re: [DISCUSS] AIP-92 Isolate DAG parsing logic

2025-08-07 Thread Jarek Potiuk
Well, you started it - so it's up to you to decide if you think we have consensus, or whether we need a vote. And It's not a question of "informal" vote but it's rather clear following the https://www.apache.org/foundation/voting.html that we either need a LAZY CONSENSUS or VOTE thread. Both are f

Re: [DISCUSS] AIP-92 Isolate DAG parsing logic

2025-08-07 Thread Sumit Maheshwari
Sorry for nudging again, but can we get into some consensus on this? I mean if this AIP isn't good enough, then we can drop it altogether and someone can rethink the whole thing. Should we do some kind of informal voting and close this thread? On Mon, Aug 4, 2025 at 3:32 PM Jarek Potiuk wrote: >

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

2025-08-07 Thread Jarek Potiuk
All good then, as the Venv operator is in the standard provider and not part of this vote Indeed. We could release it separately and manually update constraints at release time - before preparing images to use the previous version (1.4.1) of the provider. That's the power of providers :) On Thu,

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

2025-08-07 Thread Ash Berlin-Taylor
All good then, as the Venv operator is in the standard provider and not part of this vote > On 6 Aug 2025, at 23:11, Pavankumar Gopidesu wrote: > > +1 non-bindig > > Ran few example dags working fine, there is one issue with the standard > provider `PythonVirtualenvOperator` > getting an unpa

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

2025-08-07 Thread Amogh Desai
+1 binding. For both airflow-core 3.0.4 RC2 and task-sdk 1.0.4RC1: - Checked reproducible package builds - Performed SVN checks - Checked Licenses - Checked Signatures - Checked SHA512 checksums Installed the RC with breeze, tested my changes from the previous RC and ran a few example dags. Rega