Re: Elasticsearch support for Spark 3.x

I am pretty certain you need to change the write.format from “es” to “org.elasticsearch.spark.sql”Sent from my iPhoneOn 8 Sep 2023, at 03:10, Dipayan Dev wrote:++ DevOn Thu, 7 Sep 2023 at 10:22 PM, Dipayan Dev wrote:Hi, Can you please elaborate your last response? I don’

Re: Elasticsearch support for Spark 3.x

Let me try that and get back. Just wondering, if there a change in the way we pass the format in connector from Spark 2 to 3? On Fri, 8 Sep 2023 at 12:35 PM, Alfie Davidson wrote: > I am pretty certain you need to change the write.format from “es” to > “org.elasticsearch.spark.sql” > > Sent fr

Re: Elasticsearch support for Spark 3.x

@Alfie Davidson : Awesome, it worked with "“org.elasticsearch.spark.sql”" But as soon as I switched to *elasticsearch-spark-20_2.12, *"es" also worked. On Fri, Sep 8, 2023 at 12:45 PM Dipayan Dev wrote: > > Let me try that and get back. Just wondering, if there a change in the > way we pass t

Re: [VOTE] Release Apache Spark 3.5.0 (RC4)

+1 (non-binding) Thanks for driving this release! On Fri, Sep 8, 2023 at 11:29 AM Holden Karau wrote: > +1 pip installing seems to function :) > > On Thu, Sep 7, 2023 at 7:22 PM Yuming Wang wrote: > >> +1. >> >> On Thu, Sep 7, 2023 at 10:33 PM yangjie01 >> wrote: >> >>> +1 >>> >>> >>> >>> *发件

Re: [VOTE] Release Apache Spark 3.5.0 (RC4)

Sorry but I'm -1 because there exists a late-arrival correctness patch although it's not a regression. - https://issues.apache.org/jira/browse/SPARK-44805 "Data lost after union using spark.sql.parquet.enableNestedColumnVectorizedReader=true" - https://github.com/apache/spark/pull/42850 - https:/

Re: [VOTE] Release Apache Spark 3.5.0 (RC4)

+1 Thank you for driving the release! On Fri, Sep 8, 2023 at 10:12 AM Jungtaek Lim wrote: > +1 (non-binding) > > Thanks for driving this release! > > On Fri, Sep 8, 2023 at 11:29 AM Holden Karau wrote: > >> +1 pip installing seems to function :) >> >> On Thu, Sep 7, 2023 at 7:22 PM Yuming Wang

Re: [VOTE] Release Apache Spark 3.5.0 (RC4)

@Dongjoon Hyun Thank you for reporting this and for your prompt response. The vote has failed. I'll cut RC5 tonight, PST time. Dongjoon Hyun 于2023年9月8日周五 15:57写道： > Sorry but I'm -1 because there exists a late-arrival correctness patch > although it's not a regression. > > - https://issues.apa

Re: [VOTE] Release Apache Spark 3.5.0 (RC4)

This issue is not a regression and yet we fail the vote? Couldn't this issue have been fixed in 3.5.1? Sorry I am new, so maybe this is how it works? On Sat, 9 Sep 2023, 02:29 Dongjoon Hyun, wrote: > Sorry but I'm -1 because there exists a late-arrival correctness patch > although it's not a r