10:34 AM Cheng Su wrote:
> >>>>>
> >>>>>> +1 for this. This is exciting movement to efficiently read bucketed
> >>>>>> table from other systems (Hive, Trino & Presto)!
> >>>>>>
> >>>>>>
> >>
>>>>>
>>>>>>1. Is migrating Hive table read path to data source v2, being a
>>>>>>prerequisite of this SPIP?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hive table read path is currently a m
Hive table read path is currently a mix of data source v1 (for Parquet
>>>>> & ORC file format only), and legacy Hive code path (HiveTableScanExec). In
>>>>> the SPIP, I am seeing we only make change for data source v2, so wondering
>>>>>
gt; Hive table read path is currently a mix of data source v1 (for Parquet
>>>>> & ORC file format only), and legacy Hive code path (HiveTableScanExec). In
>>>>> the SPIP, I am seeing we only make change for data source v2, so wondering
>>>>
gt; >
> >
> >
> > Just curious if there’s any other use cases we are targeting as part of
> > SPIP.
> >
> >
> >
> > Thanks,
> >
> > Cheng Su
> >
> >
> >
> >
> >
> >
> >
> > *From: *Ryan Blue
&g
le is merged in master recently (
>>>> SPARK-19256 <https://issues.apache.org/jira/browse/SPARK-19256> has
>>>> details).
>>>>
>>>>
>>>>
>>>>1. Would aggregate work automatically after the SPIP?
>>>>
>>>
before aggregate. Just want to bring to our attention that it would be
>>> great to consider aggregate as well when doing this proposal.
>>>
>>>
>>>
>>>1. Any major use cases in mind except Hive bucketed table?
>>>
>>>
>>>
y major use cases in mind except Hive bucketed table?
>>
>>
>>
>> Just curious if there’s any other use cases we are targeting as part of
>> SPIP.
>>
>>
>>
>> Thanks,
>>
>> Cheng Su
>>
>>
>>
>>
>>
&
; Cheng Su
>
>
>
>
>
>
>
> *From: *Ryan Blue
> *Date: *Tuesday, October 26, 2021 at 9:39 AM
> *To: *John Zhuge
> *Cc: *Chao Sun , Wenchen Fan ,
> Cheng Su , DB Tsai , Dongjoon Hyun <
> dongjoon.h...@gmail.com>, Hyukjin Kwon , Wenchen Fan
> , angers zhu , dev
e’s any other use cases we are targeting as part of
> SPIP.
>
>
>
> Thanks,
>
> Cheng Su
>
>
>
>
>
>
>
> *From: *Ryan Blue
> *Date: *Tuesday, October 26, 2021 at 9:39 AM
> *To: *John Zhuge
> *Cc: *Chao Sun , Wenchen Fan ,
> Cheng Su , DB Tsai ,
Date: Tuesday, October 26, 2021 at 9:39 AM
To: John Zhuge
Cc: Chao Sun , Wenchen Fan , Cheng Su
, DB Tsai , Dongjoon Hyun
, Hyukjin Kwon , Wenchen Fan
, angers zhu , dev
, huaxin gao
Subject: Re: [DISCUSS] SPIP: Storage Partitioned Join for Data Source V2
Instead of commenting on the doc,
Instead of commenting on the doc, could we keep discussion here on the dev
list please? That way more people can follow it and there is more room for
discussion. Comment threads have a very small area and easily become hard
to follow.
Ryan
On Tue, Oct 26, 2021 at 9:32 AM John Zhuge wrote:
> +1
+1 Nicely done!
On Tue, Oct 26, 2021 at 8:08 AM Chao Sun wrote:
> Oops, sorry. I just fixed the permission setting.
>
> Thanks everyone for the positive support!
>
> On Tue, Oct 26, 2021 at 7:30 AM Wenchen Fan wrote:
>
>> +1 to this SPIP and nice writeup of the design doc!
>>
>> Can we open co
Oops, sorry. I just fixed the permission setting.
Thanks everyone for the positive support!
On Tue, Oct 26, 2021 at 7:30 AM Wenchen Fan wrote:
> +1 to this SPIP and nice writeup of the design doc!
>
> Can we open comment permission in the doc so that we can discuss details
> there?
>
> On Tue,
+1 on this,
Wenchen Fan 于2021年10月26日周二 下午10:29写道：
> +1 to this SPIP and nice writeup of the design doc!
>
> Can we open comment permission in the doc so that we can discuss details
> there?
>
> On Tue, Oct 26, 2021 at 8:29 PM Hyukjin Kwon wrote:
>
>> Seems making sense to me.
>>
>> Would be gre
+1 to this SPIP and nice writeup of the design doc!
Can we open comment permission in the doc so that we can discuss details
there?
On Tue, Oct 26, 2021 at 8:29 PM Hyukjin Kwon wrote:
> Seems making sense to me.
>
> Would be great to have some feedback from people such as @Wenchen Fan
> @Cheng
Seems making sense to me.
Would be great to have some feedback from people such as @Wenchen Fan
@Cheng Su @angers zhu
.
On Tue, 26 Oct 2021 at 17:25, Dongjoon Hyun wrote:
> +1 for this SPIP.
>
> On Sun, Oct 24, 2021 at 9:59 AM huaxin gao wrote:
>
>> +1. Thanks for lifting the current restri
+1 for this SPIP.
On Sun, Oct 24, 2021 at 9:59 AM huaxin gao wrote:
> +1. Thanks for lifting the current restrictions on bucket join and making
> this more generalized.
>
> On Sun, Oct 24, 2021 at 9:33 AM Ryan Blue wrote:
>
>> +1 from me as well. Thanks Chao for doing so much to get it to this
+1. Thanks for lifting the current restrictions on bucket join and making
this more generalized.
On Sun, Oct 24, 2021 at 9:33 AM Ryan Blue wrote:
> +1 from me as well. Thanks Chao for doing so much to get it to this point!
>
> On Sat, Oct 23, 2021 at 11:29 PM DB Tsai wrote:
>
>> +1 on this SPIP
+1 from me as well. Thanks Chao for doing so much to get it to this point!
On Sat, Oct 23, 2021 at 11:29 PM DB Tsai wrote:
> +1 on this SPIP.
>
> This is a more generalized version of bucketed tables and bucketed
> joins which can eliminate very expensive data shuffles when joins, and
> many use
+1 on this SPIP.
This is a more generalized version of bucketed tables and bucketed
joins which can eliminate very expensive data shuffles when joins, and
many users in the Apache Spark community have wanted this feature for
a long time!
Thank you, Ryan and Chao, for working on this, and I look f
21 matches
Mail list logo
