Re: [DISCUSS] SPIP: Monthly preview release

2025-07-01 Thread Hyukjin Kwon
> 1. Since we "release" the preview, we will go through the VOTE process. What is the expected overhead on doing this monthly? (Maybe this would be coupled with the questions below.) We have to vote for every preview every month for now. We can try and see how it goes. > 2. What quality do we for

Re: [DISCUSS] SPIP: Monthly preview release

2025-07-01 Thread Jungtaek Lim
Thanks for the proposal. The direction is awesome - we have such a long interval between minor releases and this would help us address some of the issues from the long release cadence. I'd like to understand a couple of things. 1. Since we "release" the preview, we will go through the VOTE proces

Re: [DISCUSS] SPIP: Monthly preview release

2025-07-01 Thread Hyukjin Kwon
Let me start the vote tmr if we're all good with this :-). On Wed, 2 Jul 2025 at 10:12, Anton Okolnychyi wrote: > Having monthly preview releases for Spark is going to be huge for projects > like Iceberg and Delta. > > - Anton > > On Tue, Jul 1, 2025 at 5:43 PM Dongjoon Hyun wrote: > >> Thank y

Re: [DISCUSS] SPIP: Monthly preview release

2025-07-01 Thread Anton Okolnychyi
Having monthly preview releases for Spark is going to be huge for projects like Iceberg and Delta. - Anton On Tue, Jul 1, 2025 at 5:43 PM Dongjoon Hyun wrote: > Thank you for the clarification, Hyukjin. Also, thank you for sharing your > direction, DB. > > I agree with you folks that the AS-IS

Re: [DISCUSS] SPIP: Monthly preview release

2025-07-01 Thread Dongjoon Hyun
Thank you for the clarification, Hyukjin. Also, thank you for sharing your direction, DB. I agree with you folks that the AS-IS scope of SPIP is a good start. +1 for the SPIP because `4.1.0-previewX` itself is actually very helpful already during developing Spark subprojects like "Spark Connect

Re: [DISCUSS] SPIP: Monthly preview release

2025-07-01 Thread Hyukjin Kwon
Hi Dongjoon, Thanks a lot for your detailed feedback and great questions! Let me clarify my current proposal and thoughts: 1. Regarding Spark 5.0 schedule At the moment, I don’t have a concrete Spark 5.0 schedule in mind. I included the stable major releases in the Final Success criteria mainly t

Re: [DISCUSS] SPIP: Monthly preview release

2025-07-01 Thread DB Tsai
Thank you, Hyukjin for driving the SPIP and for your work on the release automation infrastructure — it’s a huge step forward. I’ve been thinking about this topic quite a bit since the Spark 4.0 release. While Spark continues to deliver meaningful improvements in every release and enjoys acti

Re: [DISCUSS] SPIP: Monthly preview release

2025-07-01 Thread Dongjoon Hyun
Thank you so much for the suggestion and achieving the automated infra, Hyujkin. I have a few questions. 1. Since the SPIP suggests Apache Spark 5.0 ("Stable major releases") as "Q8. Final Success" criteria. I'm wondering if you have some schedule in your mind for Spark 5.0 in next 2 years? 2.

Re: [VOTE] Release Apache Spark K8s Operator 0.4.0 (RC1)

2025-07-01 Thread Liu Cao
+1 (non-binding) On Mon, Jun 30, 2025 at 11:17 PM Attila Mészáros wrote: > +1 > > On Tue, Jul 1, 2025 at 8:11 AM Yang Jie wrote: > > > > +1 > > > > On 2025/07/01 00:35:56 Wenchen Fan wrote: > > > +1 > > > > > > On Tue, Jul 1, 2025 at 6:23 AM L. C. Hsieh wrote: > > > > > > > +1 > > > > > > > >

Re: [VOTE] Release Apache Spark K8s Operator 0.4.0 (RC1)

2025-07-01 Thread Yang Jie
+1 On 2025/07/01 00:35:56 Wenchen Fan wrote: > +1 > > On Tue, Jul 1, 2025 at 6:23 AM L. C. Hsieh wrote: > > > +1 > > > > Thanks Dongjoon. > > > > On Mon, Jun 30, 2025 at 8:52 AM Dongjoon Hyun > > wrote: > > > > > > Please vote on releasing the following candidate as Apache Spark K8s > > Operat