Re: Question regarding "Insufficient number of network buffers"

2019-04-11 Thread zhijiang
Hi Allen, There are two ways for setting network buffers. The old way via `taskmanager.network.numberOfBuffers` is deprecated. The new way is via three parameters min,max and fraction. The specific formula is Math.min(network.memory.max, Math.max(network.memory.min, network.memory.fraction * j

Question regarding "Insufficient number of network buffers"

2019-04-11 Thread Xiangfeng Zhu
Hello, My name is Allen, and I'm currently researching different distributed execution engines. I wanted to run some benchmarks on Flink with a 10-node cluster(each node has 64vCPUs and 376GB memory). I ran the program with parallelism 320 and got an error message: "Caused by: java.io.IOException: