taskmanager.heap.mb defines the total amount of memory used by a task
manager. The "heap" part of the parameter name originates from the time
when Flink could not allocate off-heap and I agree it is confusing.
Nonetheless, we decided to keep it for backward compatibility.
The memory fraction is de
Thanks!
I will try this one: taskmanager.memory.size. So I should expect this will be
the off-heap memory size, right?
I am using taskmanager.heap.mb=some value, taskmanager.memory.off-heap: true
Memory usage goes up to 99%.
The documentation is confusing:
taskmanager.memory.size: The amount of
Oh yes, good point! The documentation needs to be updated to something like:
"The amount of memory (in megabytes) that the task manager reserves on-heap
or off-heap (depending on taskmanager.memory.off-heap) for sorting, hash
tables, and caching of intermediate results. If unspecified (-1), the
me
Updating the documentation will partially remove the confusion.
Heap memory is JVM managed while off-heap is outside JVM and I would define its
size separately.
In my case, I see memory usage going up to full utilisation on a node where
full RAM available is 128GB in the following conditions:
ta
Hi Ovidiu,
the parameters to configure the amount of managed memory
(taskmanager.memory.size,
taskmanager.memory.fraction) are valid for on and off-heap memory.
Have you tried these parameters and didn't they work as expected?
Best, Fabian
2016-03-16 11:43 GMT+01:00 Ovidiu-Cristian MARCU <
ovi
Hi,
Is it possible to add a parameter off-heap.size for the task manager off-heap
memory [1]?
It is not possible to limit the off-heap memory size, at least I found nothing
in the documentation.
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.0/setup/config.html#managed-memory
