Re: Re: About Memory Spilling to Disk in Flink

2021-03-24 Thread Guowei Ma
Hi, Roc Thanks for your detailed explanation. I could not find any "stream" operator that uses `ExternalSorterBuilder` by "find usage" of the IDEA. Best, Guowei On Wed, Mar 24, 2021 at 3:27 PM Roc Marshal wrote: > Hi, Guowei Ma. > As far as I know, flink writes some in-memory data to disk w

Re:Re: About Memory Spilling to Disk in Flink

2021-03-24 Thread Roc Marshal
Hi, Guowei Ma. As far as I know, flink writes some in-memory data to disk when memory is running low. I noticed that flink uses ExternalSorterBuilder for batch operations in the org.apache.flink.runtime.operator.sort package, but I'm curious to confirm if this technique is also used in strea

Re: About Memory Spilling to Disk in Flink

2021-03-24 Thread Guowei Ma
Hi, Roc Could you explain more about your question? Best, Guowei On Wed, Mar 24, 2021 at 2:47 PM Roc Marshal wrote: > Hi, > > Can someone tell me where flink uses memory spilling to write to disk? > Thank you. > > Best, Roc. > > > >

About Memory Spilling to Disk in Flink

2021-03-23 Thread Roc Marshal
Hi, Can someone tell me where flink uses memory spilling to write to disk? Thank you. Best, Roc.