Re: Flink streaming job with iterations gets stuck waiting for network buffers

Hi Andrey, As Paris has explained it, this is a known issue and there are ongoing efforts to solve it. I can suggest a workaround: limit the amount of messages sent into the iteration manually. You can do this with a e.g. a Map operator that limits records per seconds and simply sends what i

Re: Any way to increase sort buffer size?

rg/apache/flink/optimizer/Optimizer.java [2] https://github.com/apache/flink/blob/master/flink-optimizer/src/main/java/org/apache/flink/optimizer/traversals/PlanFinalizer.java 2016-11-16 17:29 GMT+01:00 Gábor Hermann <mailto:m...@gaborhermann.com>>: Hi all, Is there any way to

Any way to increase sort buffer size?

Hi all, Is there any way to increase the sort buffer size other than increasing the overall TaskManager memory? The following error comes up running a job with huge matrix block objects on a cluster: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an ex