Thanks Ken for the info.This is something which I have done when running
spark batch jobs. However, in this case I really want to understand if
there is anything wrong with the job itself. Is the flink kafka consumer or
some other piece needs more memory than I am allocating.
Hemant
On Fri, Jul 2
When we run Flink jobs in EMR (typically batch, though) we disable the pmem
(permanent memory) and vmem (virtual memory) checks.
This was initially done for much older versions of Flink (1.6???), where the
memory model wasn’t so well documented or understood by us.
But I think the pmem check mi
Hi,
I am running a streaming job (Flink 1.9) on EMR on yarn. Flink web UI or
metrics reported from prometheus shows total memory usage within specified
task manager memory - 3GB.
Metrics shows below numbers(in MB) -
Heap - 577
Non Heap - 241
DirectMemoryUsed - 852
Non-heap does rise gradually, s
