Re: [DISCUSS] Allow GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go

A good point agreed. Mich Talebzadeh, Technologist | Architect | Data Engineer | Generative AI | FinCrime PhD Imperial College London London, United Kingdom view my Linkedin profile <

Re: [DISCUSS] Allow GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go

Alright! let me start the vote! On Thu, 4 Jul 2024 at 16:31, Mich Talebzadeh wrote: > A good point agreed. > > Mich Talebzadeh, > Technologist | Architect | Data Engineer | Generative AI | FinCrime > PhD Imperial College > London

[VOTE] Allow GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go

Hi all, I’d like to start a vote for allowing GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go. Please also refer to: - Discussion thread: https://lists.apache.org/thread/tsqm0dv01f7jgkv5l4kyvtpw4tc6f420 - JIRA ticket: https://issues.apache.org/jira/br

Re: [VOTE] Allow GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go

+1 (non-binding) On Thu, Jul 4, 2024 at 19:13 Hyukjin Kwon wrote: > Hi all, > > I’d like to start a vote for allowing GitHub Actions runs for > contributors' PRs without approvals in apache/spark-connect-go. > > Please also refer to: > >- Discussion thread: > https://lists.apache.org/thread/

Re: [VOTE] Move Spark Connect server to builtin package (Client API layer stays external)

+1 John Zhuge ezt írta (időpont: 2024. júl. 4., Cs, 5:38): > +1 > > > John Zhuge > > > On Wed, Jul 3, 2024 at 7:41 PM Gengliang Wang wrote: > >> +1 >> >> On Wed, Jul 3, 2024 at 4:48 PM Reynold Xin >> wrote: >> >>> +1 >>> >>> On Wed, Jul 3, 2024 at 4:45 PM L. C. Hsieh wrote: >>> +1 >

Re: [VOTE] Allow GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go

+1 non-binding Mich Talebzadeh, Technologist | Architect | Data Engineer | Generative AI | FinCrime PhD Imperial College London London, United Kingdom view my Linkedin profile

[DISCUSS] Auto scaling support for structured streaming

Hi, I remember there was a discussion about better supporting auto scaling for structured streaming. Is there anything happening with that for the upcoming Spark 4.0 release? Will there be support for auto scaling (at least on K8s) spark structured streaming apps? Thanks, Nimrod

Re: [VOTE] Allow GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go

+1 Although given its a US holiday maybe keep the vote open for an extra day? Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 YouTube Live Streams: https://www.youtube.com/user/holdenkarau On Thu,

Re: [VOTE] Allow GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go

+1 (non-binding) On Thu, Jul 4, 2024 at 7:15 PM Holden Karau wrote: > +1 > > Although given its a US holiday maybe keep the vote open for an extra day? > > Twitter: https://twitter.com/holdenkarau > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9

Re: [VOTE] Move Spark Connect server to builtin package (Client API layer stays external)

(I will leave this vote open till 10th July, considering that its holiday season in US) On Thu, 4 Jul 2024 at 23:39, Peter Toth wrote: > +1 > > John Zhuge ezt írta (időpont: 2024. júl. 4., Cs, > 5:38): > >> +1 >> >> >> John Zhuge >> >> >> On Wed, Jul 3, 2024 at 7:41 PM Gengliang Wang wrote: >>

Re: [VOTE] Allow GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go

(I will leave this vote open till 10th July, considering that its holiday season in US) On Fri, 5 Jul 2024 at 06:12, Martin Grund wrote: > +1 (non-binding) > > On Thu, Jul 4, 2024 at 7:15 PM Holden Karau > wrote: > >> +1 >> >> Although given its a US holiday maybe keep the vote open for an extr

Re: Spark decommission

Hi Rajesh, We use it production at scale. We run spark on kubernetes on aws cloud and here are the key things that we do 1) we run driver on on-demand node 2) we have configured decommission along with fallback option on to S3, try the latest single zone S3 for this. 3) We use pvc aware scheduling