Issue Upgrading to 3.2

2021-10-29 Thread Adam Binford
Hi devs, I'm working on getting Apache Sedona upgraded to work with Spark 3.2, and ran into a weird issue I wanted to get some feedback on. The PR and current discussion can be found here: https://github.com/apache/incubator-sedona/pull/557 To try to sum up in a quick way, this library defines cu

Re: [DISCUSS] SPIP: Storage Partitioned Join for Data Source V2

2021-10-29 Thread L . C . Hsieh
Thanks all for your inputs here! Seems the discussion already settles, I will be the shepherd for the SPIP and call for a vote on the SPIP moving forward in a new thread. On 2021/10/28 13:05:53, Wenchen Fan wrote: > Thanks for the explanation! It makes sense to always resolve the logical > t

[VOTE] SPIP: Storage Partitioned Join for Data Source V2

2021-10-29 Thread L . C . Hsieh
Hi all, I’d like to start a vote for SPIP: Storage Partitioned Join for Data Source V2. The proposal is to support a new type of join: storage partitioned join which covers bucket join support for DataSourceV2 but is more general. The goal is to let Spark leverage distribution properties reported

Re: [VOTE] SPIP: Storage Partitioned Join for Data Source V2

2021-10-29 Thread L . C . Hsieh
I'll start with my +1. On 2021/10/29 17:30:03, L. C. Hsieh wrote: > Hi all, > > I’d like to start a vote for SPIP: Storage Partitioned Join for Data Source > V2. > > The proposal is to support a new type of join: storage partitioned join which > covers bucket join support for DataSourceV2 b

Re: [VOTE] SPIP: Storage Partitioned Join for Data Source V2

2021-10-29 Thread Russell Spitzer
+1 This is a great idea, (I have no Apache Spark voting points) On Fri, Oct 29, 2021 at 12:41 PM L. C. Hsieh wrote: > > I'll start with my +1. > > On 2021/10/29 17:30:03, L. C. Hsieh wrote: > > Hi all, > > > > I’d like to start a vote for SPIP: Storage Partitioned Join for Data > Source V2. > >

Re: [VOTE] SPIP: Storage Partitioned Join for Data Source V2

2021-10-29 Thread Dongjoon Hyun
+1 Dongjoon On 2021/10/29 17:48:59, Russell Spitzer wrote: > +1 This is a great idea, (I have no Apache Spark voting points) > > On Fri, Oct 29, 2021 at 12:41 PM L. C. Hsieh wrote: > > > > > I'll start with my +1. > > > > On 2021/10/29 17:30:03, L. C. Hsieh wrote: > > > Hi all, > > > > > >

Re: [VOTE] SPIP: Storage Partitioned Join for Data Source V2

2021-10-29 Thread huaxin gao
+1 On Fri, Oct 29, 2021 at 10:59 AM Dongjoon Hyun wrote: > +1 > > Dongjoon > > On 2021/10/29 17:48:59, Russell Spitzer > wrote: > > +1 This is a great idea, (I have no Apache Spark voting points) > > > > On Fri, Oct 29, 2021 at 12:41 PM L. C. Hsieh wrote: > > > > > > > > I'll start with my +1.

Re: [VOTE] SPIP: Storage Partitioned Join for Data Source V2

2021-10-29 Thread Ryan Blue
+1 On Fri, Oct 29, 2021 at 11:06 AM huaxin gao wrote: > +1 > > On Fri, Oct 29, 2021 at 10:59 AM Dongjoon Hyun > wrote: > >> +1 >> >> Dongjoon >> >> On 2021/10/29 17:48:59, Russell Spitzer >> wrote: >> > +1 This is a great idea, (I have no Apache Spark voting points) >> > >> > On Fri, Oct 29, 2

Re: [VOTE] SPIP: Storage Partitioned Join for Data Source V2

2021-10-29 Thread DB Tsai
+1 DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1 On Fri, Oct 29, 2021 at 11:42 AM Ryan Blue wrote: > +1 > > On Fri, Oct 29, 2021 at 11:06 AM huaxin gao > wrote: > >> +1 >> >> On Fri, Oct 29, 2021 at 10:59 AM Dongjoon Hyun >> wrote: >> >>> +1 >>> >>> Dongjoon >>> >>> On 2021/10/

Re: [VOTE] SPIP: Storage Partitioned Join for Data Source V2

2021-10-29 Thread Holden Karau
+1 On Fri, Oct 29, 2021 at 3:07 PM DB Tsai wrote: > +1 > > DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1 > > > On Fri, Oct 29, 2021 at 11:42 AM Ryan Blue wrote: > >> +1 >> >> On Fri, Oct 29, 2021 at 11:06 AM huaxin gao >> wrote: >> >>> +1 >>> >>> On Fri, Oct 29, 2021 at 10:59 AM

Re: [VOTE] SPIP: Storage Partitioned Join for Data Source V2

2021-10-29 Thread Cheng Su
+1 Thanks, Cheng Su From: Holden Karau Date: Friday, October 29, 2021 at 12:41 PM To: DB Tsai Cc: Dongjoon Hyun , Ryan Blue , dev , huaxin gao Subject: Re: [VOTE] SPIP: Storage Partitioned Join for Data Source V2 +1 On Fri, Oct 29, 2021 at 3:07 PM DB Tsai wrote: +1 DB Tsai | https://www.