Great :-)
On Tue, Jun 8, 2021 at 1:11 PM Yingjie Cao wrote:
> Hi Till,
>
> Thanks for the suggestion. The blog post is already on the way.
>
> Best,
> Yingjie
>
> Till Rohrmann 于2021年6月8日周二 下午5:30写道：
>
>> Thanks for the update Yingjie. Would it make sense to write a short blog
>> post about thi
Hi Till,
Thanks for the suggestion. The blog post is already on the way.
Best,
Yingjie
Till Rohrmann 于2021年6月8日周二 下午5:30写道：
> Thanks for the update Yingjie. Would it make sense to write a short blog
> post about this feature including some performance improvement numbers? I
> think this could
Thanks for the update Yingjie. Would it make sense to write a short blog
post about this feature including some performance improvement numbers? I
think this could be interesting to our users.
Cheers,
Till
On Mon, Jun 7, 2021 at 4:49 AM Jingsong Li wrote:
> Thanks Yingjie for the great effort!
Thanks Yingjie for the great effort!
This is really helpful to Flink Batch users!
Best,
Jingsong
On Mon, Jun 7, 2021 at 10:11 AM Yingjie Cao wrote:
> Hi devs & users,
>
> The FLIP-148[1] has been released with Flink 1.13 and the final
> implementation has some differences compared with the ini
Hi devs & users,
The FLIP-148[1] has been released with Flink 1.13 and the final
implementation has some differences compared with the initial proposal in
the FLIP document. To avoid potential misunderstandings, I have updated the
FLIP document[1] accordingly and I also drafted another document[2]
Hi devs,
This discussion thread has been opened for over a week. If there is no
other concerns, I'd like to open a voting thread soon.
Best,
Yingjie
Yingjie Cao 于2020年10月23日周五 上午11:56写道：
> Hi Zhijiang,
>
> Thanks for your reply and suggestions.
>
> 1. For
> taskmanager.network.sort-merge-block
Hi Zhijiang,
Thanks for your reply and suggestions.
1. For
taskmanager.network.sort-merge-blocking-shuffle.max-files-per-partition, we
decide to append all data produced by one result partition to one file, so
this option will be removed.
2. For
taskmanager.network.sort-merge-blocking-shuffle.bu
Thanks for launching the discussion and the respective FLIP, Yingjie!
In general, I am +1 for this proposal since sort-merge ability has already been
taken widely in other batch-based project, like MR, Spark, etc.
And it indeed has some performance benefits in some scenarios as mentioned in
FLIP
Thanks for sharing the preliminary numbers with us Yingjie. The numbers
look quite impressive :-)
Cheers,
Till
On Thu, Oct 15, 2020 at 5:25 PM Yingjie Cao wrote:
> Hi Till,
>
> Thanks for your reply and comments.
>
> You are right, the proposed sort-merge based shuffle is an extension of the
>
Hi Till,
Thanks for your reply and comments.
You are right, the proposed sort-merge based shuffle is an extension of the
existing blocking shuffle and does not change any default behavior of Flink.
As for the performance, according to our previous experience, sort-merge
based implementation can
Hi Yingjie,
thanks for proposing the sort-merge based blocking shuffle. I like the
proposal and it does not seem to change the internals of Flink. Instead it
is an extension of existing interfaces which makes it a
non-invasive addition.
Do you have any numbers comparing the performance of the sor
11 matches
Mail list logo
