Hello,
I would like to ask you something regarding an error I’m facing running java
code over the cluster at DIMA.
Caused by: java.io.IOException: Insufficient number of network buffers:
required 300, but only 161 available. The total number of network buffers is
currently set to 70480. You can increase this number by setting the
configuration key 'taskmanager.network.numberOfBuffers'.
at
org.apache.flink.runtime.io.network.buffer.NetworkBufferPool.createBufferPool(NetworkBufferPool.java:176)
at
org.apache.flink.runtime.io.network.NetworkEnvironment.registerTask(NetworkEnvironment.java:313)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:464)
at java.lang.Thread.run(Thread.java:853)
The error seems pretty clear so as suggested, I increased the
taskmanager.network.numberOfBuffers from the flink-conf.yaml, (first was setted
to 35240), but actually this didn’t solve the problem, and the number of the
available buffers is almost always the same.
Could you explain about some further configurations I should check above the
cluster?
Besides solve this issue I would be interested in understanding why I get this
and where these values come from.
Thanks in advance for your time,
Guido
