In our usage of Flink, our Yarn Job Manager never goes above ~48 MB of heap 
utilization. In order to maximize the heap available to the Task Managers I 
thought we could shrink our Job Manager heap setting down from the 1024MB we 
were using to something tiny like 128MB. However, doing so results in the 
runtime error:

java.lang.IllegalArgumentException: The JobManager memory (64) is below the 
minimum required memory amount of 768 MB
at 
org.apache.flink.yarn.AbstractYarnClusterDescriptor.setJobManagerMemory(AbstractYarnClusterDescriptor.java:187)
…

Looking into it: this value isn’t controlled by the settings in yarn-site.xml 
but is actually hardcoded in Flink code base to 768 MB. (see 
AbstractYarnDescriptor.java where MIN_JM_MEMORY = 768.)


Why is this hardcoded?
Why not let value be set via the Yarn Site Configuration xml?
Why such a high minimum?


Thanks,
Dan

Reply via email to