Hello, How many memory your yarn containers are configured to have ? This error may be due to running a flink on yarn cluster with more memory than you have in containers. Could you check it, and maybe set containers memory to a more suitable value ?
regards 2016-06-01 1:22 GMT+02:00 prateekarora <prateek.arora...@gmail.com>: > Hi > > I am running flink 1.0.2 with Yarn . > > After running application for some time , Yarn kill my container due to > running beyond physical memory limits . > > how can i debug memory issue ? > > below are the logs : > > Container container_1463184272818_0165_01_000012 is completed with > diagnostics: Container > [pid=19349,containerID=container_1463184272818_0165_01_000012] is running > beyond physical memory limits. Current usage: 6.0 GB of 6 GB physical > memory > used; 9.1 GB of 12.6 GB virtual memory used. Killing container. > > Dump of the process-tree for container_1463184272818_0165_01_000012 : > |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE > |- 19356 19349 19349 19349 (java) 39350 9110 9711140864 1581168 > /usr/lib/jvm/java-7-oracle-cloudera/bin/java -Xms4608m -Xmx4608m > -XX:MaxDirectMemorySize=4608m > -Djava.library.path=/home/nativelibraries/native_lib/ > > -Dlog.file=/var/log/hadoop-yarn/container/application_1463184272818_0165/container_1463184272818_0165_01_000012/taskmanager.log > -Dlogback.configurationFile=file:logback.xml > -Dlog4j.configuration=file:log4j.properties > org.apache.flink.yarn.YarnTaskManagerRunner --configDir . > |- 19349 19345 19349 19349 (bash) 0 0 11456512 359 /bin/bash -c > /usr/lib/jvm/java-7-oracle-cloudera/bin/java -Xms4608m -Xmx4608m > -XX:MaxDirectMemorySize=4608m > -Djava.library.path=/home/nativelibraries/native_lib/ > > -Dlog.file=/var/log/hadoop-yarn/container/application_1463184272818_0165/container_1463184272818_0165_01_000012/taskmanager.log > -Dlogback.configurationFile=file:logback.xml > -Dlog4j.configuration=file:log4j.properties > org.apache.flink.yarn.YarnTaskManagerRunner --configDir . 1> > > /var/log/hadoop-yarn/container/application_1463184272818_0165/container_1463184272818_0165_01_000012/taskmanager.out > 2> > > /var/log/hadoop-yarn/container/application_1463184272818_0165/container_1463184272818_0165_01_000012/taskmanager.err > > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > > > > -- > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/yarn-kill-container-due-to-running-beyond-physical-memory-limits-How-can-i-debug-memory-issue-tp7296.html > Sent from the Apache Flink User Mailing List archive. mailing list archive > at Nabble.com. > -- Alexis Gendronneau alexis.gendronn...@corp.ovh.com a.gendronn...@gmail.com
