> On 9 Feb 2016, at 06:53, Sean Owen <so...@cloudera.com> wrote: > > > I think you can let YARN over-commit RAM though, and allocate more > memory than it actually has. It may be beneficial to let them all > think they have an extra GB, and let one node running the AM > technically be overcommitted, a state which won't hurt at all unless > you're really really tight on memory, in which case something might > get killed.
from my test VMs <property> <description>Whether physical memory limits will be enforced for containers. </description> <name>yarn.nodemanager.pmem-check-enabled</name> <value>false</value> </property> <property> <name>yarn.nodemanager.vmem-check-enabled</name> <value>false</value> </property> it does mean that a container can swap massively, hurting the performance of all containers around it as IO bandwidth gets soaked up —which is why the checks are on for shared clusters. If it's dedicated, you can overcommit