Alright, thanks for the feedback. I also agree with it. Then this is resolved.
> On 19 Mar 2020, at 14:14, Till Rohrmann wrote:
>
> I agree with Xintong's proposal. If we see that many users run into this
> problem, then one could think about escalating the warning message into a
> failure.
>
>
I agree with Xintong's proposal. If we see that many users run into this
problem, then one could think about escalating the warning message into a
failure.
Cheers,
Till
On Thu, Mar 19, 2020 at 4:23 AM Xintong Song wrote:
> I think recommend a minimum value in docs and throw a warning if the hea
I think recommend a minimum value in docs and throw a warning if the heap
size is too small should be good enough.
Not sure about failing job if the min heap is not fulfilled. As already
mentioned, it would be hard to determine the min heap size. And if we make
the min heap configurable, then in an
Hi all,
One thing more thing to mention, the current calculations can lead to
arbitrary small JVM Heap, maybe even zero.
I suggest to introduce a check where we at least recommend to set the JVM
heap to e.g. 128Mb.
Additionally, we can demand some minimum value to function and fail if it
is not f
Hi all,
Thanks for the feedback, Xintong and Till.
> rename jobmanager.memory.direct.size into jobmanager.memory.off-heap.size
I am ok with that to align it with TM and avoid further complications for
users.
I will adjust the FLIP.
> change the default value of JM Metaspace size to 256 MB
Inde
Thanks for creating this FLIP Andrey.
I agree with Xintong that we should rename jobmanager.memory.direct.size
into jobmanager.memory.off-heap.size which accounts for native and direct
memory usage. I think it should be good enough and is easier to understand
for the user.
Concerning the default
Thanks Andrey for kicking this discussion off.
Regarding "direct" vs. "off-heap", I'm personally in favor of renaming the
"direct" memory in the current FLIP-116[1] to "off-heap" memory, and making
it also account for user native memory usage.
On one hand, I think it would be good that JM & TM pr
Hi All,
As you may have noticed, 1.10 release included an extensive improvements to
memory management and configuration of Task Managers, FLIP-49: [1]. The
memory configuration of Job Managers has not been touched in 1.10.
Although, Job Manager's memory model does not look so sophisticated as
for
