Re: [VOTE] New Spark Connect Client Repository for Rust

Sounds great, thanks for the clarification. +1 (non-binding) On Tue, May 20, 2025 at 3:20 PM Renjie Liu wrote: > Hi, Jungtaek, Yuanjian: > > Yes, I'm more than happy to contribute to this project! > > On Tue, May 20, 2025 at 2:14 PM Yuanjian Li > wrote: > >> Jungtaek - That’s definitely a reas

Re: [VOTE] New Spark Connect Client Repository for Rust

Hi, Jungtaek, Yuanjian: Yes, I'm more than happy to contribute to this project! On Tue, May 20, 2025 at 2:14 PM Yuanjian Li wrote: > Jungtaek - That’s definitely a reasonable question—and one I had asked > earlier as well. Renjie and Steve will work together in the early stages to > handle auth

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 On 2025/05/20 04:54:50 Yuming Wang wrote: > +1 > > On Tue, May 20, 2025 at 12:19 PM Szehon Ho wrote: > > > +1 (non-binding) > > > > Checked signature, checksum, ran basic tests on spark-4.0.0-bin-hadoop3 > > Thanks > > Szehon > > > > > > > > On Mon, May 19, 2025 at 9:07 PM Denny Lee wrote:

Re: [DISCUSS] Apache Spark 3.5.6 (automated release)

+1, It would be truly awesome if we could use GitHub Actions for release. On 2025/05/20 05:08:16 Wenchen Fan wrote: > +1, look forward to Spark's first automated release! > > On Tue, May 20, 2025 at 12:55 PM Yuming Wang wrote: > > > +1. > > > > On Tue, May 20, 2025 at 12:51 PM Hyukjin Kwon > >

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 (non-binding) Thanks, Cheng Pan > On May 20, 2025, at 12:54, Yuming Wang wrote: > > +1 > > On Tue, May 20, 2025 at 12:19 PM Szehon Ho > wrote: >> +1 (non-binding) >> >> Checked signature, checksum, ran basic tests on spark-4.0.0-bin-hadoop3 >> Thanks >> S

Re: [DISCUSS] Apache Spark 3.5.6 (automated release)

+1, look forward to Spark's first automated release! On Tue, May 20, 2025 at 12:55 PM Yuming Wang wrote: > +1. > > On Tue, May 20, 2025 at 12:51 PM Hyukjin Kwon > wrote: > >> Hi, >> >> I am testing automatic release via Github Actions ( >> https://github.com/apache/spark/pull/50911), and it's a

Re: [DISCUSS] Apache Spark 3.5.6 (automated release)

+1. On Tue, May 20, 2025 at 12:51 PM Hyukjin Kwon wrote: > Hi, > > I am testing automatic release via Github Actions ( > https://github.com/apache/spark/pull/50911), and it's almost getting > there. > > I would like to release the actual release with this GitHub Actions also > for testing purpos

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 On Tue, May 20, 2025 at 12:19 PM Szehon Ho wrote: > +1 (non-binding) > > Checked signature, checksum, ran basic tests on spark-4.0.0-bin-hadoop3 > Thanks > Szehon > > > > On Mon, May 19, 2025 at 9:07 PM Denny Lee wrote: > >> +1 (non-binding) >> >> On Mon, May 19, 2025 at 9:02 PM Rozov, Vlad

[DISCUSS] Apache Spark 3.5.6 (automated release)

Hi, I am testing automatic release via Github Actions ( https://github.com/apache/spark/pull/50911), and it's almost getting there. I would like to release the actual release with this GitHub Actions also for testing purposes. WDYT guys?

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 (non-binding) Checked signature, checksum, ran basic tests on spark-4.0.0-bin-hadoop3 Thanks Szehon On Mon, May 19, 2025 at 9:07 PM Denny Lee wrote: > +1 (non-binding) > > On Mon, May 19, 2025 at 9:02 PM Rozov, Vlad > wrote: > >> +1 (non-binding) >> >> Vlad >> >> On May 19, 2025, at 8:56 

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 (non-binding) On Mon, May 19, 2025 at 9:02 PM Rozov, Vlad wrote: > +1 (non-binding) > > Vlad > > On May 19, 2025, at 8:56 PM, Jules Damji wrote: > > + 1 (non-binding) > — > Sent from my iPhone > Pardon the dumb thumb typos :) > > On May 19, 2025, at 5:26 PM, Gengliang Wang wrote: > >  > +1

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 (non-binding) Vlad On May 19, 2025, at 8:56 PM, Jules Damji wrote: + 1 (non-binding) — Sent from my iPhone Pardon the dumb thumb typos :) On May 19, 2025, at 5:26 PM, Gengliang Wang wrote:  +1 On Mon, May 19, 2025 at 5:21 PM Jungtaek Lim mailto:kabhwan.opensou...@gmail.com>> wrote: +1

Re: [VOTE] Release Spark 4.0.0 (RC7)

+ 1 (non-binding) —Sent from my iPhonePardon the dumb thumb typos :)On May 19, 2025, at 5:26 PM, Gengliang Wang wrote:+1On Mon, May 19, 2025 at 5:21 PM Jungtaek Lim wrote:+1 (non-binding)On Tue, May 20, 2025 at 8:47 AM Ruifeng Zheng wrote:+1On

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 Thank you, Wenchen. Dongjoon. On Mon, May 19, 2025 at 5:27 PM Gengliang Wang wrote: > +1 > > On Mon, May 19, 2025 at 5:21 PM Jungtaek Lim > wrote: > >> +1 (non-binding) >> >> On Tue, May 20, 2025 at 8:47 AM Ruifeng Zheng >> wrote: >> >>> +1 >>> >>> On Tue, May 20, 2025 at 7:04 AM Hyukjin

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 On Mon, May 19, 2025 at 5:21 PM Jungtaek Lim wrote: > +1 (non-binding) > > On Tue, May 20, 2025 at 8:47 AM Ruifeng Zheng wrote: > >> +1 >> >> On Tue, May 20, 2025 at 7:04 AM Hyukjin Kwon >> wrote: >> >>> +1 >>> >>> On Mon, 19 May 2025 at 21:27, Wenchen Fan wrote: >>> Same as before, I

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 (non-binding) On Tue, May 20, 2025 at 8:47 AM Ruifeng Zheng wrote: > +1 > > On Tue, May 20, 2025 at 7:04 AM Hyukjin Kwon wrote: > >> +1 >> >> On Mon, 19 May 2025 at 21:27, Wenchen Fan wrote: >> >>> Same as before, I'll start with my own +1. >>> >>> On Mon, May 19, 2025 at 8:25 PM Wenchen Fa

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 On Tue, May 20, 2025 at 7:04 AM Hyukjin Kwon wrote: > +1 > > On Mon, 19 May 2025 at 21:27, Wenchen Fan wrote: > >> Same as before, I'll start with my own +1. >> >> On Mon, May 19, 2025 at 8:25 PM Wenchen Fan wrote: >> >>> Please vote on releasing the following candidate as Apache Spark vers

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 On Mon, 19 May 2025 at 21:27, Wenchen Fan wrote: > Same as before, I'll start with my own +1. > > On Mon, May 19, 2025 at 8:25 PM Wenchen Fan wrote: > >> Please vote on releasing the following candidate as Apache Spark version >> 4.0.0. >> >> The vote is open until May 22 (PST) and passes if

Re: [VOTE] New Spark Connect Client Repository for Rust

Just one question - who will be in charge of maintaining the new repo? The committers and PMC members could help on merging, but who will author the code and who will review? Also how many active committers + PMC members are familiar with Rust language? On Tue, May 20, 2025 at 4:15 AM Yuanjian Li

Re: [VOTE] New Spark Connect Client Repository for Rust

For reference, we are carrying forward the votes from the discussion thread: (*** = binding) *+1:* Jules Damji Yuanjian Li * Huaxin Gao * Peter Toth Renjie Liu Denny Lee Niranjan Jayakar *+0: *None *-1:*None Yuanjian Li ： > Please vote on the proposal to introduce a new Apache Sp

[VOTE] New Spark Connect Client Repository for Rust

Please vote on the proposal to introduce a new Apache Spark repository for a Spark Connect Client in Rust. This vote will remain open for the next 72 hours and will pass if it receives a majority of +1 PMC votes, with a minimum of 3 binding +1 votes. As discussed in the previous thread: https://li

Re: [DISCUSS] New Spark Connect Client repository for Rust language

Agreed. Let me start a voting thread for this and move the existing +1s to the new thread. Mridul Muralidharan 于2025年5月19日周一 08:44写道： > > Please start an explicit vote thread - so that it is not confusing. > > This shows us as a DISCUSS thread currently. > > Regards, > Mridul > > On Mon, May 19,

Re: [DISCUSS] New Spark Connect Client repository for Rust language

Please start an explicit vote thread - so that it is not confusing. This shows us as a DISCUSS thread currently. Regards, Mridul On Mon, May 19, 2025 at 6:44 AM Niranjan Jayakar wrote: > Thank you Renji and Fokko for your responses. Let's move forward with a > separate repo then. > > +1. > > O

Re: Structured streaming consumer group offset management in case of consumption of topic with same name from different Kafka clusters

Spark does not rely on Kafka's commit, in fact it tracks the stream progress itself and reads via offsets (e.g. from last read points). Why do you want to commit? On Mon, May 19, 2025 at 5:58 PM megh vidani wrote: > Hello Spark Dev Community, > > Reaching out for the below problem statement. > >

Re: [VOTE] Release Spark 4.0.0 (RC7)

Same as before, I'll start with my own +1. On Mon, May 19, 2025 at 8:25 PM Wenchen Fan wrote: > Please vote on releasing the following candidate as Apache Spark version > 4.0.0. > > The vote is open until May 22 (PST) and passes if a majority +1 PMC votes > are cast, with a minimum of 3 +1 votes

[VOTE] Release Spark 4.0.0 (RC7)

Please vote on releasing the following candidate as Apache Spark version 4.0.0. The vote is open until May 22 (PST) and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 4.0.0 [ ] -1 Do not release this package because ... To le

Re: [DISCUSS] New Spark Connect Client repository for Rust language

Thank you Renji and Fokko for your responses. Let's move forward with a separate repo then. +1. On Mon, May 19, 2025 at 1:12 PM Denny Lee wrote: > +1 (non-binding) > > > On Mon, May 19, 2025 at 02:42 Renjie Liu wrote: > >> +1 （non-binding) >> >> Hi, Niranjan: >> >> We also had a discussion whe

Re: [DISCUSS] New Spark Connect Client repository for Rust language

+1 (non-binding) On Mon, May 19, 2025 at 02:42 Renjie Liu wrote: > +1 （non-binding) > > Hi, Niranjan: > > We also had a discussion when starting different > language implementations(too old and I can't find the link to the original > thread), and eventually we decided to use separate repos. It

Re: [DISCUSS] New Spark Connect Client repository for Rust language

+1 （non-binding) Hi, Niranjan: We also had a discussion when starting different language implementations(too old and I can't find the link to the original thread), and eventually we decided to use separate repos. It turns out to work quite well. Among all the benefits, the most important one is