For me, it's -0. Yes. Likely some people used it, but we made it terribly difficult for our Helm chart users to use it. We have no support for multi-executors in the Helm Chart we released. I would only be for making it stable if we release Helm Chart with multi-executors as a first-class citizen. Even a doc on how to use it with the current Helm chart would be enough.
On Fri, Feb 14, 2025 at 9:15 PM Jens Scheffler <j_scheff...@gmx.de.invalid> wrote: > +1 marking it stable - actually I am a bit surprised - we are using this > since months in production :-D So we can confirm it is rock solid, not > only stable. > > I am not sure whether we really come to a 2.11... > > In regards of Deprecation: All hybrid executors are in cncf-k8s provider > package. We can deprecate independent of Airflow 3 and clear the > documentation. Or also we could build a "balcony" == check into these > executors that they are raising an Exception when somebody attempts to > configure/start them in Airflow 3 (and leave them supported until > support for 2.x ends in the provider package) > > On 14.02.25 19:17, Oliveira, Niko wrote: > > Thanks for the reply and the support Constance! > > > > > > Fair call out on moving multi-exec to stable in 2.11.0 release. We could > just deprecate the hardcoded hybrid executors and leave it at that for > 2.11, however to me it feels like there should be something the community > sees that is stable for them to migrate to if we're telling them to move > away from the old hybrid executors. > > > > Calling it stable just means the public interface won't change in a > breaking way and that they can depend on this feature existing into the > future instead of being removed at a short notice. We can still collect > usage data and fix issues if we find them like we do any other stable > feature in Airflow. > > > > Thanks again for weighing in, and I'm curious to hear what others think > as well! > > > > Cheers, > > > > Niko > > > > > > > > ________________________________ > > From: Constance Martineau <consta...@astronomer.io.INVALID> > > Sent: Friday, February 14, 2025 9:27:47 AM > > To: dev@airflow.apache.org > > Subject: RE: [EXT] [DISCUSS] Mark Multi Exec Config as stable and remove > old hardcoded hybrid execs > > > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous > ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas > certain que le contenu ne présente aucun risque. > > > > > > > > Hey Niko, > > > > Thanks for kicking off this discussion! > > > > I’m fully on board with removing support for hardcoded hybrid executors > in > > favor of multi-executor support for 3.0—it’s a great step forward in > > simplifying execution models. Adding deprecation warnings in 2.11 also > > makes sense to give users a smooth transition period. > > > > Regarding marking multi-executor support stable in 2.11: The only thing > I’d > > be mindful of is ensuring we get enough real-world testing across > different > > multi-executor combinations before marking it fully stable. We haven’t > seen > > a ton of feedback on that front yet, and I’d rather avoid unexpected > > surprises post-release. > > > > Also, since we’re focused on wrapping up 3.0, I’d prefer to keep changes > in > > 2.11 minimal where possible. Deprecation warnings are a great addition, > but > > I’d want to avoid creating any expectations that we’ll be patching 2.11 > > specifically for this feature now that it’s considered stable. > > > > Curious to hear what others think! > > > > Cheers, > > Constance > > > > On Thu, Feb 13, 2025 at 8:16 PM Oliveira, Niko > <oniko...@amazon.com.invalid> > > wrote: > > > >> Hello folks! > >> > >> Multiple Executor Configuration (aka hybrid executors, AIP-61) has been > >> out a while now and we haven't had many issues with it. I'm proposing we > >> mark it as stable. > >> > >> Relatedly I would also like to discuss removing support for the older > >> hardcoded hybrid executors (e.g. CeleryKubernetesExecutor, etc) in > Airflow > >> 3. We still have many lines of code in core Airflow that are coupled > >> directly to these executors since they need special handling. They also > >> don't track changes to the base executor very well and so they tend to > >> break every time that interface changes [2]. I figure Airflow 3.0 might > be > >> the best opportunity we'll have in a while to remove these executors. > >> > >> We of course don't have to remove them from the provider packages they > >> live in, so new versions of providers will still be compatible with pre > 3.0 > >> Airflow to use those executors. > >> > >> I propose that we mark Multiple Executor Configuration stable in 2.11 > and > >> clearly mark the old hardcoded hybrid executors as deprecated also in > 2.11. > >> This gives time for folks to migrate if they'd like. Then in 3.0 remove > all > >> our special handling and testing from core Airflow relating to these > >> hardcoded hybrid executors. > >> > >> What do y'all think? > >> > >> Cheers, > >> Niko > >> > >> [1] - https://github.com/apache/airflow/pull/46742 > >> [2] - https://github.com/apache/airflow/pull/41602 > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >
