Re: [VOTE] Release Apache Spark 3.5.5 deprecating `spark.databricks.*` configuration

+1 (non-binding) On Wed, Feb 19, 2025 at 2:22 PM Reynold Xin wrote: > +1 > > On Wed, Feb 19, 2025 at 2:02 PM Zhou Jiang wrote: > >> >> +1 (non-binding) >> >> On Feb 19, 2025, at 04:20, Peter Toth wrote: >> >>  >> +1 >> >> On Wed, Feb 19, 2025 at 10:20 AM Max Gekk wrote: >> >>> +1 >>> >>> On

Re: [VOTE] Release Apache Spark 3.5.5 deprecating `spark.databricks.*` configuration

+1 On Wed, Feb 19, 2025 at 2:02 PM Zhou Jiang wrote: > > +1 (non-binding) > > On Feb 19, 2025, at 04:20, Peter Toth wrote: > >  > +1 > > On Wed, Feb 19, 2025 at 10:20 AM Max Gekk wrote: > >> +1 >> >> On Wed, Feb 19, 2025 at 9:15 AM L. C. Hsieh wrote: >> >>> +1 >>> >>> On Tue, Feb 18, 2025 at

Re: [VOTE] Release Apache Spark 3.5.5 deprecating `spark.databricks.*` configuration

+1 (non-binding)On Feb 19, 2025, at 04:20, Peter Toth wrote:+1On Wed, Feb 19, 2025 at 10:20 AM Max Gekk wrote:+1On Wed, Feb 19, 2025 at 9:15 AM L. C. Hsieh wrote:+1 On Tue, Feb 18, 2025 at 9:46 PM dongjoon.hyun wrote: > > Please v

Re: [VOTE] Release Apache Spark 3.5.5 deprecating `spark.databricks.*` configuration

+1 On Wed, Feb 19, 2025 at 10:20 AM Max Gekk wrote: > +1 > > On Wed, Feb 19, 2025 at 9:15 AM L. C. Hsieh wrote: > >> +1 >> >> On Tue, Feb 18, 2025 at 9:46 PM dongjoon.hyun >> wrote: >> > >> > Please vote to deprecate `spark.databricks.*` configuration at Apache >> Spark 3.5.5. >> > This is a p

Re: Extending Spark with a custom ExternalClusterManager

Hi devs, Let me pull some spark-submit developers into this discussion. @dongjoon-hyun @HyukjinKwon @cloud-fan What are your thoughts on making spark-submit fully and generically support ExternalClusterManager implementations? The current situation is that the only way to submit a Spark job vi

Re: [VOTE] Release Spark 4.0.0 (RC1)

+1 Dr Mich Talebzadeh, Architect | Data Science | Financial Crime | Forensic Analysis | GDPR view my Linkedin profile On Wed, 19 Feb 2025 at 09:31, Wenchen Fan wrote: > Please vote on releasing the following candidate as Apache

[VOTE] Release Spark 4.0.0 (RC1)

Please vote on releasing the following candidate as Apache Spark version 4.0.0. The vote is open until February 21 (PST) and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 4.0.0 [ ] -1 Do not release this package because ...

Re: [VOTE] Release Apache Spark 3.5.5 deprecating `spark.databricks.*` configuration

+1 On Wed, Feb 19, 2025 at 9:15 AM L. C. Hsieh wrote: > +1 > > On Tue, Feb 18, 2025 at 9:46 PM dongjoon.hyun > wrote: > > > > Please vote to deprecate `spark.databricks.*` configuration at Apache > Spark 3.5.5. > > This is a part of the following on-going discussion. > > > > - DISCUSSION: > htt

Re: Deprecating and banning `spark.databricks.*` config from Apache Spark repository

Looks like we are concerned about having the vendor's name in the config as much as security concerns or so, although we would make it still be graceful for a single version line. If that's the case, I'm OK with removing the config in 4.0.0 and taking the path of the migration doc, while deprecati

Re: [VOTE] Release Apache Spark 3.5.5 deprecating `spark.databricks.*` configuration

+1 On Tue, Feb 18, 2025 at 9:46 PM dongjoon.hyun wrote: > > Please vote to deprecate `spark.databricks.*` configuration at Apache Spark > 3.5.5. > This is a part of the following on-going discussion. > > - DISCUSSION: https://lists.apache.org/thread/qwxb21g5xjl7xfp4rozqmg1g0ndfw2jd > (Deprecat

Re: [VOTE] Release Apache Spark 3.5.5 deprecating `spark.databricks.*` configuration

+1 (non-binding) Deprecating has not been a problem for me, and removal can be said in another thread. On Wed, Feb 19, 2025 at 4:26 PM Mich Talebzadeh wrote: > +1 > > Dr Mich Talebzadeh, > Architect | Data Science | Financial Crime | Forensic Analysis | GDPR > >view my Linkedin profile >