Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 binding On Fri, Aug 2, 2024 at 8:30 AM Wei Lee wrote: > +1 binding. > > Best, > Wei > > > On Aug 2, 2024, at 5:06 AM, Vishnu Chilukoori > wrote: > > > > +1 non-binding > > > > On Thu, Aug 1, 2024 at 2:22 PM Ankit Chaurasia > wrote: > > > >> +1 non-binding > >> > >> *Ankit Chaurasia* > >> >

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 binding. Best, Wei > On Aug 2, 2024, at 5:06 AM, Vishnu Chilukoori > wrote: > > +1 non-binding > > On Thu, Aug 1, 2024 at 2:22 PM Ankit Chaurasia wrote: > >> +1 non-binding >> >> *Ankit Chaurasia* >> >> >> >> On Fri, 2 Aug 2024 at 01:45, Vishnu Chilukoori >> wrote: >> >>> +1 >>> >

Re: AWS Open Source Credits Program

Awesome! > On Jul 29, 2024, at 3:08 AM, Utkarsh Sharma > wrote: > > Sounds great! :) > > On Sun, Jul 28, 2024 at 4:17 PM Pierre Jeambrun > wrote: > >> Great news! >> >> On Sat 27 Jul 2024 at 01:15, Sadha Chilukoori >> wrote: >> >>> That's wonderful news. >>> >>> On Fri, Jul 26, 2024 at 1

Re: [VOTE] AIP-76: Asset Partitions

+1 (binding) Best, Wei > On Aug 2, 2024, at 9:09 AM, Kaxil Naik wrote: > > We should resolve the open comments on the AIP before we conclude the vote > though > > On Fri, 2 Aug 2024 at 02:09, Kaxil Naik wrote: > >> +1 binding >> >> On Thu, 1 Aug 2024 at 18:07, Buğra Öztürk wrote: >> >>> +

Re: [RESULT][VOTE] July 2024 PR of the Month

Yep. Grats! On Fri, Aug 2, 2024 at 2:52 AM Pavankumar Gopidesu wrote: > Congratulations Niko 🎉🎉. > > Regards, > Pavan > > On Fri, Aug 2, 2024 at 1:45 AM Vikram Koka > wrote: > > > Very cool! Congratulations Niko! > > > > > > On Thu, Aug 1, 2024 at 5:42 PM Kaxil Naik wrote: > > > > > Congrats N

Re: [VOTE] AIP-76: Asset Partitions

We should resolve the open comments on the AIP before we conclude the vote though On Fri, 2 Aug 2024 at 02:09, Kaxil Naik wrote: > +1 binding > > On Thu, 1 Aug 2024 at 18:07, Buğra Öztürk wrote: > >> +1 non-binding >> >> On Thu, 1 Aug 2024, 16:07 Brent Bovenzi, >> wrote: >> >> > +1 (binding) >

Re: [VOTE] AIP-76: Asset Partitions

+1 binding On Thu, 1 Aug 2024 at 18:07, Buğra Öztürk wrote: > +1 non-binding > > On Thu, 1 Aug 2024, 16:07 Brent Bovenzi, > wrote: > > > +1 (binding) > > > > On Thu, Aug 1, 2024 at 3:10 AM Tzu-ping Chung > > wrote: > > > > > Hi all, > > > > > > Kicking start the vote on the final data awarenes

Re: [DISCUSS] AIP-81: Enhanced Security in CLI via Integration of API

Thanks, Bugra, great turnaround on such a short notice. I have added my comments too. Regards, Kaxil On Sat, 27 Jul 2024 at 17:08, Buğra Öztürk wrote: > Thanks, Vikram! Answered them in the proposal. > --- > Thanks, Jarek! It's a pleasure to be part of and work on this one! :) > > Thanks for th

Re: [RESULT][VOTE] July 2024 PR of the Month

Congratulations Niko 🎉🎉. Regards, Pavan On Fri, Aug 2, 2024 at 1:45 AM Vikram Koka wrote: > Very cool! Congratulations Niko! > > > On Thu, Aug 1, 2024 at 5:42 PM Kaxil Naik wrote: > > > Congrats Niko > > > > On Fri, 2 Aug 2024 at 00:00, Ferruzzi, Dennis > > > > > wrote: > > > > > Congrats Nik

Re: [DISCUSS] External event driven scheduling in Airflow

Thanks Vincent and Kaxil, Agree it can be added to future work. Looking at discussion on event mapping, Today we know the source systems and we are aware of the schemas from source systems (S3, SQS, Event Arc etc;) However, as new services emerge and their schemas evolve, this may change. Agree

Re: [RESULT][VOTE] July 2024 PR of the Month

Very cool! Congratulations Niko! On Thu, Aug 1, 2024 at 5:42 PM Kaxil Naik wrote: > Congrats Niko > > On Fri, 2 Aug 2024 at 00:00, Ferruzzi, Dennis > > wrote: > > > Congrats Niko, and almost Vincent :P > > > > - ferruzzi > > > > > > > > From: Briana Okyere > >

Re: [RESULT][VOTE] July 2024 PR of the Month

Congrats Niko On Fri, 2 Aug 2024 at 00:00, Ferruzzi, Dennis wrote: > Congrats Niko, and almost Vincent :P > > - ferruzzi > > > > From: Briana Okyere > Sent: Wednesday, July 31, 2024 3:11 PM > To: dev@airflow.apache.org > Subject: RE: [EXT] [RESULT][VOTE] July 20

Re: [DISCUSS] External event driven scheduling in Airflow

Yes but the big difference is you will create a single user for EventBridge since that is the one sending request to Airflow, single user for EventArc for GCP and 1 user for every other EventListener or application as compared to 1 user per type of Payload (since Airflow will need to understan

Re: [DISCUSS] External event driven scheduling in Airflow

Ah, I didn't notice this discussion on the email thread, which I was commenting on the wiki page. I am comfortable with the poll-based approach, but struggling with the push-based approach. My concerns are generally consistent with the ones raised here. I see the mapping/conversion function approa

Re: [DISCUSS] External event driven scheduling in Airflow

I am all for it - if we want to stick to event bridge or similar and recommend it to our users, it's perfectly fine for me, It would be great however to add a documentation explaining the steps and some examples - ideally for most of our providers and "standard" ways of triggering such an event. Th

Re: [DISCUSSION] AIP-83 Remove Execution Date Unique Constraint from DAG Run

Replied inline > On 2 Aug 2024, at 03:52, Daniel Standish > wrote: > > Hi @Tzu-ping Chung > > I just had a chance to look over the AIP after noticing that the vote thread > started. > > I have a few questions / things to note. It looks like this is more than >

Re: [DISCUSS] External event driven scheduling in Airflow

I was discussing this with Vincent. In either case, same as now or the one proposed in the AIP, a user will have to use something like AWS EventBridge[1] or GCS EventArc [2] where users will consume the event from object storage (S3 object creation for example), and then they will have to add the A

Re: [DISCUSS] External event driven scheduling in Airflow

I proposed the mapping - because it's the easiest way (I think) to map between the "native" source to "airflow" target expectations. There are many producers of such events, and Airflow is the consumer. And it seems appropriate to have a way for our users to easily plug events produced from one sys

Re: [VOTE] AIP-79 & AIP-84 Remove Flask AppBuilder as a Core Dependency & UI REST API

+1 - with the same concerns as Kaxil. On Fri, Aug 2, 2024 at 1:02 AM Kaxil Naik wrote: > +1 binding on both AIPs. But I want to make sure that we add Connection > form decoupling to AIP-79 (or other AIP) unless we rely on FAB for > backwards compatibility. > > On Thu, 1 Aug 2024 at 23:49, Scheff

Re: [DISCUSS] Extendable parsing controls (a.k.a. Pluggable Parsers)

Small update: I've created the page on wiki: https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-85+Extendable+DAG+parsing+controls. I suggest moving inline-comments there instead of the originally linked doc. On Wed, Jul 31, 2024 at 1:57 AM Kaxil Naik wrote: > Hi Igor, > > I have added perm

Re: [VOTE] AIP-79 & AIP-84 Remove Flask AppBuilder as a Core Dependency & UI REST API

+1 binding on both AIPs. But I want to make sure that we add Connection form decoupling to AIP-79 (or other AIP) unless we rely on FAB for backwards compatibility. On Thu, 1 Aug 2024 at 23:49, Scheffler Jens (XC-AS/EAE-ADA-T) wrote: > +1 binding for both (lost overview about all the votes - did

Re: [RESULT][VOTE] July 2024 PR of the Month

Congrats Niko, and almost Vincent :P - ferruzzi From: Briana Okyere Sent: Wednesday, July 31, 2024 3:11 PM To: dev@airflow.apache.org Subject: RE: [EXT] [RESULT][VOTE] July 2024 PR of the Month CAUTION: This email originated from outside of the organization. Do

Re: [DISCUSS] External event driven scheduling in Airflow

I would love for VOTE to get started on this one. I think most of the commenters and those who replied to this email are happy with the proposal on the poll-based approach. Regarding the push-based approach, I am not convinced that the proposed implementation has any gains over what's already avai

RE: [VOTE] AIP-79 & AIP-84 Remove Flask AppBuilder as a Core Dependency & UI REST API

+1 binding for both (lost overview about all the votes - did I already? 😃) Mit freundlichen Grüßen / Best regards Jens Scheffler Alliance: Enabler - Tech Lead (XC-AS/EAE-ADA-T) Robert Bosch GmbH | Hessbruehlstraße 21 | 70565 Stuttgart-Vaihingen | GERMANY | www.bosch.com Tel. +49 711 811-91508 |

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 non-binding On Thu, Aug 1, 2024 at 2:22 PM Ankit Chaurasia wrote: > +1 non-binding > > *Ankit Chaurasia* > > > > On Fri, 2 Aug 2024 at 01:45, Vishnu Chilukoori > wrote: > > > +1 > > > > On Thu, Aug 1, 2024 at 12:09 AM Tzu-ping Chung > > > wrote: > > > > > Hi, > > > > > > This is the actual

Re: [VOTE] AIP-67 - Multi-team deplyment of Airflow Components

Voting ends a week from now - i.e Thursday 8th of August 2023 (midnight). On Thu, Aug 1, 2024 at 10:13 PM Jarek Potiuk wrote: > Hello here, > > I reviewed and - I hope - addressed all the comments so far. One most > recent change in the doc was to allow a connection/variable to belong to > multi

Re: [VOTE] AIP-79 & AIP-84 Remove Flask AppBuilder as a Core Dependency & UI REST API

+1 for both, non-binding On Thu, Aug 1, 2024 at 9:34 PM Igor Kholopov wrote: > +1 (non-binding, both) > > On Thu, Aug 1, 2024 at 9:04 PM Vikram Koka > wrote: > > > +1 binding on both AIP-79 and AIP-84. > > > > Vikram > > > > > > On Thu, Aug 1, 2024 at 11:35 AM Vincent Beck > wrote: > > > > > +

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 non-binding *Ankit Chaurasia* On Fri, 2 Aug 2024 at 01:45, Vishnu Chilukoori wrote: > +1 > > On Thu, Aug 1, 2024 at 12:09 AM Tzu-ping Chung > wrote: > > > Hi, > > > > This is the actual vote thread for AIP-83 > > > > AIP-83 Remove Execution Date Unique Constraint from DAG Run > > https://

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 On Thu, Aug 1, 2024 at 12:09 AM Tzu-ping Chung wrote: > Hi, > > This is the actual vote thread for AIP-83 > > AIP-83 Remove Execution Date Unique Constraint from DAG Run > https://cwiki.apache.org/confluence/x/SA3OEg > > Discussion thread: > https://lists.apache.org/thread/8rokf8ogp7g5pf0koqk

[VOTE] AIP-67 - Multi-team deplyment of Airflow Components

Hello here, I reviewed and - I hope - addressed all the comments so far. One most recent change in the doc was to allow a connection/variable to belong to multiple teams. Also I mentioned a potential future change that will allow us to run "per-team" schedulers - leaving the UI/DB schema as the ma

Re: [DISCUSSION] AIP-83 Remove Execution Date Unique Constraint from DAG Run

Hi @Tzu-ping Chung I just had a chance to look over the AIP after noticing that the vote thread started. I have a few questions / things to note. It looks like this is more than simply removing the unique constraint, but actually following through on removing execution date entirely, and replac

[ANNOUNCE] Apache Airflow Newsletter July 2024

Hey All, The July Issue of the Apache Airflow Newsletter is out! Please read it here Or subscribe here to get it directly in your inbox < https://apache.us14.list-manage.com/subscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa > -- Brian

Apache Airflow 2.10.0b2 available for testing

Hey, fellow Airflowers, I have cut Airflow 2.10.0b2 now that all the main features have been included. This "snapshot" is intended for members of the Airflow developer community to test the build and allow early testing of 2.10.0. Please test this beta and create GitHub issues wherever possible i

Re: [VOTE] AIP-79 & AIP-84 Remove Flask AppBuilder as a Core Dependency & UI REST API

+1 (non-binding, both) On Thu, Aug 1, 2024 at 9:04 PM Vikram Koka wrote: > +1 binding on both AIP-79 and AIP-84. > > Vikram > > > On Thu, Aug 1, 2024 at 11:35 AM Vincent Beck wrote: > > > +1 binding for both > > > > On 2024/08/01 18:33:51 Jed Cunningham wrote: > > > +1 binding! > > > > > > > --

Re: [VOTE] AIP-79 & AIP-84 Remove Flask AppBuilder as a Core Dependency & UI REST API

+1 binding on both AIP-79 and AIP-84. Vikram On Thu, Aug 1, 2024 at 11:35 AM Vincent Beck wrote: > +1 binding for both > > On 2024/08/01 18:33:51 Jed Cunningham wrote: > > +1 binding! > > > > - > To unsubscribe, e-mail: dev-un

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 (binding) I love any and all AIPs to remove deprecated code in the next major release! ❤ From: Brent Bovenzi Sent: Thursday, August 1, 2024 7:05:35 AM To: dev@airflow.apache.org Subject: RE: [EXT] [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG

Re: [VOTE] AIP-79 & AIP-84 Remove Flask AppBuilder as a Core Dependency & UI REST API

+1 binding for both On 2024/08/01 18:33:51 Jed Cunningham wrote: > +1 binding! > - To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org

Re: [VOTE] AIP-79 & AIP-84 Remove Flask AppBuilder as a Core Dependency & UI REST API

+1 binding!

[VOTE] AIP-79 & AIP-84 Remove Flask AppBuilder as a Core Dependency & UI REST API

Hi, Jed and I prepared two AIPs to flesh out how to finish AIP-38 Modern Web Application for Airflow 3.0. We would like to open it up for a vote. AIP-79 https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-79%3A+Remove+Flask+AppBuilder+as+Core+dependency AIP-84 UI REST API https://cwiki.apach

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 binding

Re: [VOTE] AIP-76: Asset Partitions

+1 non-binding On Thu, 1 Aug 2024, 16:07 Brent Bovenzi, wrote: > +1 (binding) > > On Thu, Aug 1, 2024 at 3:10 AM Tzu-ping Chung > wrote: > > > Hi all, > > > > Kicking start the vote on the final data awareness AIP targeting 3.0. > > > > AIP-76 Asset Partitions > > https://cwiki.apache.org/confl

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 non-binding On Thu, 1 Aug 2024, 19:03 Utkarsh Sharma, wrote: > +1 binding > > Thanks, > Utkarsh Sharma > > On Thu, Aug 1, 2024 at 10:10 PM Ephraim Anierobi < > ephraimanier...@gmail.com> > wrote: > > > +1 > > > > On Thu, 1 Aug 2024 at 17:40, Kaxil Naik wrote: > > > > > +1 binding > > > > > >

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 binding Thanks, Utkarsh Sharma On Thu, Aug 1, 2024 at 10:10 PM Ephraim Anierobi wrote: > +1 > > On Thu, 1 Aug 2024 at 17:40, Kaxil Naik wrote: > > > +1 binding > > > > On Thu, 1 Aug 2024 at 15:34, Pavankumar Gopidesu < > gopidesupa...@gmail.com> > > wrote: > > > > > +1 (non-binding) > > > >

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 On Thu, 1 Aug 2024 at 17:40, Kaxil Naik wrote: > +1 binding > > On Thu, 1 Aug 2024 at 15:34, Pavankumar Gopidesu > wrote: > > > +1 (non-binding) > > > > On Thu, Aug 1, 2024 at 3:22 PM Vincent Beck wrote: > > > > > +1 binding > > > > > > On 2024/08/01 14:05:35 Brent Bovenzi wrote: > > > > +1

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 binding On Thu, 1 Aug 2024 at 15:34, Pavankumar Gopidesu wrote: > +1 (non-binding) > > On Thu, Aug 1, 2024 at 3:22 PM Vincent Beck wrote: > > > +1 binding > > > > On 2024/08/01 14:05:35 Brent Bovenzi wrote: > > > +1 (binding) > > > > > > On Thu, Aug 1, 2024 at 2:12 AM Tzu-ping Chung > > > >

Re: [DISCUSS] External event driven scheduling in Airflow

I agree with both of you that it is indeed a good idea and that it can be added in Future work -- doesn't need to be part of this AIP. Thanks for the interest. I was not aware of such feature and this looks > really cool! I definitely think that can be useful for Airflow, especially > for testing

Re: [DISCUSS] External event driven scheduling in Airflow

Hey Pavan, Thanks for the interest. I was not aware of such feature and this looks really cool! I definitely think that can be useful for Airflow, especially for testing when you can easily replay events received in the past. However, I do not think it should be part of the AIP and, as you ment

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 (non-binding) On Thu, Aug 1, 2024 at 3:22 PM Vincent Beck wrote: > +1 binding > > On 2024/08/01 14:05:35 Brent Bovenzi wrote: > > +1 (binding) > > > > On Thu, Aug 1, 2024 at 2:12 AM Tzu-ping Chung > > wrote: > > > > > Oops, forgot to attach an end date /facepalm > > > > > > Let’s do the mini

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 binding On 2024/08/01 14:05:35 Brent Bovenzi wrote: > +1 (binding) > > On Thu, Aug 1, 2024 at 2:12 AM Tzu-ping Chung > wrote: > > > Oops, forgot to attach an end date /facepalm > > > > Let’s do the minimum since there seems to be not much concerns behind this. > > > > The vote will run until

Re: [VOTE] AIP-76: Asset Partitions

+1 (binding) On Thu, Aug 1, 2024 at 3:10 AM Tzu-ping Chung wrote: > Hi all, > > Kicking start the vote on the final data awareness AIP targeting 3.0. > > AIP-76 Asset Partitions > https://cwiki.apache.org/confluence/x/2QyTEg > > Discussion thread > https://lists.apache.org/thread/3d80921801j6p23

Re: [VOTE] AIP-83 Remove Execution Date Unique Constraint from DAG Run

+1 (binding) On Thu, Aug 1, 2024 at 2:12 AM Tzu-ping Chung wrote: > Oops, forgot to attach an end date /facepalm > > Let’s do the minimum since there seems to be not much concerns behind this. > > The vote will run until 4th Aug 2024, 6:09 UTC. > > TP > > > > On 1 Aug 2024, at 14:08, Tzu-ping Ch

Re: [DISUSSION] AIP-78 & AIP 84 Remove FAB as core dependency and UI REST API

Yes, AIP-79 to "Remove Flask AppBuilder as a Core Dependency". There have been so many AIPs recently it's hard to keep track! On Wed, Jul 31, 2024 at 7:55 PM Vikram Koka wrote: > Thanks Jed and Brent, both of these seem quite straightforward. > > I am really glad to see the AIP-79 Remove Flask A

[VOTE] AIP-76: Asset Partitions

Hi all, Kicking start the vote on the final data awareness AIP targeting 3.0. AIP-76 Asset Partitions https://cwiki.apache.org/confluence/x/2QyTEg Discussion thread https://lists.apache.org/thread/3d80921801j6p23x3xgq6yy09gy8yvm4 The vote will run until Thursday, 2024-08-08, 08:00 UTC. This mak