Re: New blog post published - Sort-Based Blocking Shuffle Implementation in Flink

2021-11-08 Thread Zhilong Hong
Thank you for writing this blog post, Daisy and Kevin! It helps me to understand what sort-based shuffle is and how to use it. Looking forward to your future improvements! On Wed, Nov 3, 2021 at 6:32 PM Yuxin Tan wrote: > Thanks Daisy and Kevin! The IO scheduling idea of the sequential reading >

Re: New blog post published - Sort-Based Blocking Shuffle Implementation in Flink

2021-11-03 Thread Yuxin Tan
Thanks Daisy and Kevin! The IO scheduling idea of the sequential reading and the benchmark result look really great! Looking forward to the next work. Best, Yuxin weijie guo 于2021年11月3日周三 下午5:24写道: > It's really an amazing job to fill in the defects of flink in batch > shuffle. I really appre

Re: New blog post published - Sort-Based Blocking Shuffle Implementation in Flink

2021-11-03 Thread weijie guo
It's really an amazing job to fill in the defects of flink in batch shuffle. I really appreciate the work done in io scheduling, the sequential reading of the shuffle reader can greatly improve the disk IO performance and stability. Sort-based shuffle realizes this feature in a concise and efficien

Re: New blog post published - Sort-Based Blocking Shuffle Implementation in Flink

2021-11-03 Thread Lijie Wang
Thanks Daisy and Kevin for bringing this blog, it is very helpful for understanding the principle of sort shuffle. Best, Lijie Guowei Ma 于2021年11月3日周三 下午2:57写道: > > Thank Daisy& Kevin much for your introduction to the improvement of TM > blocking shuffle, credit base+io scheduling is indeed a

Re: New blog post published - Sort-Based Blocking Shuffle Implementation in Flink

2021-11-02 Thread Guowei Ma
Thank Daisy& Kevin much for your introduction to the improvement of TM blocking shuffle, credit base+io scheduling is indeed a very interesting thing. At the same time, I look forward to this as a default setting for tm blocking shuffle. Best, Guowei On Wed, Nov 3, 2021 at 2:46 PM Gen Luo wrote

Re: New blog post published - Sort-Based Blocking Shuffle Implementation in Flink

2021-11-02 Thread Gen Luo
Thanks Daisy and Kevin! The benchmark results look really exciting! On Tue, Nov 2, 2021 at 4:38 PM David Morávek wrote: > Thanks Daisy and Kevin for a great write up! ;) Especially the 2nd part > was really interesting, I really like the idea of the single spill file > with a custom scheduling o

Re: New blog post published - Sort-Based Blocking Shuffle Implementation in Flink

2021-11-02 Thread David Morávek
Thanks Daisy and Kevin for a great write up! ;) Especially the 2nd part was really interesting, I really like the idea of the single spill file with a custom scheduling of read requests. Best, D. On Mon, Nov 1, 2021 at 10:01 AM Daisy Tsang wrote: > Hey everyone, we have a new two-part post publ