Re: [DISCUSS] Automation of RC email

We can always invalidate the vote with -1 in case it is found to be sent incorrectly ... As long as the automation does not end up generating a tonne of mails, that is, it should be fairly manageable :) I am in favor of automating it with option 1. Thanks for driving this Hyukjin ! Regards, Mri

[DISCUSS] Automation of RC email

Hi all, As some of you may know, I’ve been working on automating the Spark release process (release.yml ). The basic steps are done, and I’m now looking into automating some of the remaining manual tasks. One such task is sending the

Re: [DISCUSS] Proposal to Add Theta and Tuple Sketches to Spark SQL

Hi Lee, Thanks for the info. I'm familiar with theta and tuple sketches' implementations under the hood. Will let you know if I have any questions! Thank you for all the work your team does. Chris

[ANNOUNCE] Apache Spark Kubernetes Operator 0.3.0 released

Hi All. We are happy to announce the availability of Apache Spark Kubernetes Operator 0.3.0! - Notable Changes * Built and tested with Apache Spark 4.0 and Spark Connect Swift Client * Running on Java 24 * Promoting CRDs to v1beta1 from v1alpha1 - Website * https://s.apache.org/spark-kub

Re: [DISCUSS] Proposal to Add Theta and Tuple Sketches to Spark SQL

Hi, This is Lee Rhodes (lee...@apache.org) from the Apache DataSketches team. I am pleased that there is interest in the Spark community for integrating our library more tightly into Spark! I would like to help if I can. Unfortunately, I am not Spark fluent so I'm not going to be very useful fo

Re: [VOTE] SPIP: Real-Time Mode in Apache Spark Structured Streaming

And great effort by you Jerry to drive this proposal through. Let us see how it progresses.Will be interesting Dr Mich Talebzadeh, Architect | Data Science | Financial Crime | Forensic Analysis | GDPR view my Linkedin profile On

Re: [VOTE] SPIP: Real-Time Mode in Apache Spark Structured Streaming

Thank you all! Glad to see this much interest and support for this initiative! On Wed, Jun 4, 2025 at 1:27 PM L. C. Hsieh wrote: > Hi all, > > Thanks all for participating and your support! The vote has been passed. > I'll send out the result in a separate thread. > > On Mon, Jun 2, 2025 at 7:5

[VOTE][RESULT] SPIP: Real-Time Mode in Apache Spark Structured Streaming

The vote passes with 24 +1s (12 binding +1s), 1 +0s (1 binding +0s) and no -1s. Thanks to all who helped with the vote! (* = binding) +1: Dongjoon Hyun (*) Yuanjian Li (*) Tathagata Das (*) Huaxin Gao (*) Xiao Li (*) DB Tsai (*) L.C. Hsieh (*) Denny Lee Gengliang Wang (*) Sakthi Anish Shrigondeka

Re: [VOTE] SPIP: Real-Time Mode in Apache Spark Structured Streaming

Hi all, Thanks all for participating and your support! The vote has been passed. I'll send out the result in a separate thread. On Mon, Jun 2, 2025 at 7:53 PM Wenchen Fan wrote: > > +1 > > On Tue, Jun 3, 2025 at 10:16 AM bo yang wrote: >> >> +1 (non-binding) >> >> On Mon, Jun 2, 2025 at 7:13 PM

[VOTE][RESULT] Release Apache Spark K8s Operator 0.3.0 (RC1)

The vote passes with 12 +1s (7 binding +1s). Thanks to all who helped with the release! (* = binding) +1: - Zhou Jiang - Huaxin Gao * - Liang-Chi Hsieh * - Peter Toth - Yang Jie * - DB Tsai * - Wenchen Fan * - Liu Cao - Vlad Rozov - Dongjoon Hyun * - Jungtaek Lim - Xinrong Meng * +0: None -1: No

Re: [VOTE] Release Apache Spark K8s Operator 0.3.0 (RC1)

Thank you all. This vote passed. I'll conclude this one. Dongjoon. On 2025/06/03 00:45:13 Xinrong Meng wrote: > +1 > > Thanks Dongjoon! > > On Mon, Jun 2, 2025 at 4:29 PM Jungtaek Lim > wrote: > > > +1 (non-binding) > > > > On Mon, Jun 2, 2025 at 11:18 PM Dongjoon Hyun wrote: > > > >> +1 >

[VOTE][RESULT] Release Apache Spark Connect Swift Client 0.3.0 (RC1)

The vote passes with 15 +1s (9 binding +1s). Thanks to all who helped with the release. (* = binding) +1: - Zhou Jiang - Huaxin Gao * - Liang-Chi Hsieh * - Peter Toth - Yang Jie * - DB Tsai * - Yuanjian Li * - Wenchen Fan * - Vlad Rozov - Dongjoon Hyun * - Sandy Ryza - Denny Lee - Jungtaek Lim - X

[ANNOUNCE] Apache Spark Connect Swift Client 0.3.0 released

Hi All. We are happy to announce the availability of Apache Spark Connect Swift Client 0.3.0! This is the first release tested with the official Apache Spark 4.0.0. Website - https://apache.github.io/spark-connect-swift/ Release Note - https://github.com/apache/spark-connect-swift/releases/t

Re: [VOTE] Release Apache Spark Connect Swift Client 0.3.0 (RC1)

Thank you all. This vote passed. I'll conclude this. Dongjoon. On 2025/06/03 00:45:20 Reynold Xin wrote: > +1 > > On Mon, Jun 2, 2025 at 5:45 PM Xinrong Meng wrote: > > > +1 > > > > Thank you Dongjoon! > > > > On Mon, Jun 2, 2025 at 4:29 PM Jungtaek Lim > > wrote: > > > >> +1 (non-binding) >

Re: Question Regarding Spark Dependencies in Scala

Yes, that was my point. If I'm directly using something or not - it is really there, so it would be beneficial for me to have a way of knowing what are the exact dependencies that I have even if I don't use them directly in case a or b- because they are there. For instance, if I am creating a libr