Re: [DISCUSS] Mark Multi Exec Config as stable and remove old hardcoded hybrid execs

For me, it's -0. Yes. Likely some people used it, but we made it terribly difficult for our Helm chart users to use it. We have no support for multi-executors in the Helm Chart we released. I would only be for making it stable if we release Helm Chart with multi-executors as a first-class citizen.

Re: [DISCUSS] Mark Multi Exec Config as stable and remove old hardcoded hybrid execs

Hey Niko, Thanks for kicking off this discussion! I’m fully on board with removing support for hardcoded hybrid executors in favor of multi-executor support for 3.0—it’s a great step forward in simplifying execution models. Adding deprecation warnings in 2.11 also makes sense to give users a smoo

Re: [DISCUSS] Mark Multi Exec Config as stable and remove old hardcoded hybrid execs

Thanks for the feedback Jens, glad to hear you're using multiple executors smoothly :) > In regards of Deprecation: All hybrid executors are in cncf-k8s provider > package. We can deprecate independent of Airflow 3 and clear the > documentation. I'm less interested in marking them deprecated i

Re: [DISCUSS] Mark Multi Exec Config as stable and remove old hardcoded hybrid execs

+1 marking it stable - actually I am a bit surprised - we are using this since months in production :-D So we can confirm it is rock solid, not only stable. I am not sure whether we really come to a 2.11... In regards of Deprecation: All hybrid executors are in cncf-k8s provider package. We can

[DISCUSS] Next steps in provider's sub-projects separation

Hello here, I wanted to draw a bit of attention to the PR I've been brewing this week (I had quite a bit less time this week due to my friend's visit but I am finally catching up :). https://github.com/apache/airflow/pull/46608 I finally got it green after quite some iterations- and as explained

Re: [DISCUSS]: Proposing a "common message queue" abstraction

+1 for sure -- Regards, Vishnu Chilukoori On Fri, Feb 14, 2025, 01:12 Mehta, Shubham wrote: > Aah, thanks for clarifying Vikram. I somehow missed it as I didn't see a > corresponding hook. If you've got it covered, then all good from my end. > > Shubham > > On 2025-02-13, 3:31 PM, "Vikram Kok

Re: [DISCUSS]: Proposing a "common message queue" abstraction

+1 for the idea *Ankit Chaurasia* On Sat, 15 Feb 2025 at 04:00, Vishnu Chilukoori wrote: > +1 for sure > > > -- > Regards, > Vishnu Chilukoori > > On Fri, Feb 14, 2025, 01:12 Mehta, Shubham > wrote: > > > Aah, thanks for clarifying Vikram. I somehow missed it as I didn't see a > > corres

Re: [DISCUSS]: Proposing a "common message queue" abstraction

+1, looking forward to it. Regards, Rahul On Thu, 13 Feb, 2025, 03:00 Vikram Koka, wrote: > Hey everyone, > > I created the following (very early) draft PR > to introduce and socialize > the concept of a "Common Message Queue" abstraction, similar

Re: [DISCUSS]: Proposing a "common message queue" abstraction

Aah, thanks for clarifying Vikram. I somehow missed it as I didn't see a corresponding hook. If you've got it covered, then all good from my end. Shubham On 2025-02-13, 3:31 PM, "Vikram Koka" mailto:vik...@astronomer.io.inva>LID> wrote: CAUTION: This email originated from outside of the organ

Re: Updating "zombie task" terminology to "task heartbeat timeout"

+1 for - * No zombies * Change from Airflow 3 * Shubham's proposal to drop scheduler from the proposed new variable name: scheduler_task_heartbeat_timeout_threshold Thanks, Kunal Bhattacharya On Fri, Feb 14, 2025 at 11:20 AM Amogh Desai wrote: > Thanks for sparking this discussion, Karen. > >

Re: [DISCUSS] Mark Multi Exec Config as stable and remove old hardcoded hybrid execs

Thanks for the reply and the support Constance! Fair call out on moving multi-exec to stable in 2.11.0 release. We could just deprecate the hardcoded hybrid executors and leave it at that for 2.11, however to me it feels like there should be something the community sees that is stable for them