Hi everybody,
increasing the akka.ask.timeout solved the second issue. Anyway that was a
warning about a congestioned network. So I worked to improve the algorithm.
Increasing the numberOfBuffers and the corresponding size solved the first
issue, thus now I can run with the full DOP.
In my case ena
Other than increasing the ask.timeout, we've seen such failures being
caused by long GC pauses over bigger heaps. In such a case, you could
fiddle with a) enabling object reuse, or b) enabling off-heap memory (i.e.
taskmanager.memory.off-heap == true) to mitigate GC-induced issues a bit.
Hope it h
OK, looks like you can easily give more memory to the network stack,
e.g. for 2 GB set
taskmanager.network.numberOfBuffers = 65536
taskmanager.network.bufferSizeInBytes = 32768
For the other exception, your logs confirm that there is something
else going on. Try increasing the akka ask timeout:
Hi Ufuk,
so the memory available per node is 48294 megabytes per node, but I reserve
28 by flink conf file.
taskmanager.heap.mb = 28672
taskmanager.memory.fraction = 0.7
taskmanager.network.numberOfBuffers = 32768
taskmanager.network.bufferSizeInBytes = 16384
Anyway Follows what I found in log fi
Hey Andrea! Sorry for the bad user experience.
Regarding the network buffers: you should be able to run it after
increasing the number of network buffers, just account for it when
specifying the heap size etc. You currently allocate 32768 * 16384
bytes = 512 MB for them. If you have a very long pi
