thanks Marcelo Vanzin I see. Org.apache.spark.executor.CoarseGrainedExecutorBackend will be executed
Thanks 2014-08-21 12:34 GMT+08:00 Marcelo Vanzin <[email protected]>: > That command line you mention in your e-mail doesn't look like > something started by Spark. Spark would start one of > ApplicationMaster, ExecutableRunner or CoarseGrainedSchedulerBackend, > not "org.apache.hadoop.mapred.YarnChild". > > On Wed, Aug 20, 2014 at 6:56 PM, centerqi hu <[email protected]> wrote: >> Spark memory settings let me very misunderstanding. >> >> My code is as follows. >> >> spark-1.0.2-bin-2.4.1/bin/spark-submit --class SimpleApp \ >> --master yarn \ >> --deploy-mode cluster \ >> --queue sls_queue_1 \ >> --num-executors 3 \ >> --driver-memory 6g \ >> --executor-memory 10g \ >> --executor-cores 5 \ >> target/scala-2.10/simple-project_2.10-1.0.jar \ >> /user/www/abc/output/2014-08-*/* >> >> >> Set executor-memory 10g, >> But, see the nodemanager java process. >> Why is Xmx3072m instead of 10G? >> >> jdk1.7.0_67//bin/java -Djava.net.preferIPv4Stack=true >> -Dhadoop.metrics.log.level=WARN -Xmx3072m >> -Djava.io.tmpdir=/data/hadoop/nodemanager/usercache/www/appcache/application_1408182086233_0013/container_1408182086233_0013_01_000004/tmp >> -Dlog4j.configuration=container-log4j.properties >> -Dyarn.app.container.log.dir=/data/hadoop/logs/nodemanager/application_1408182086233_0013/container_1408182086233_0013_01_000004 >> -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA >> org.apache.hadoop.mapred.YarnChild 10.1.13.11 35389 >> attempt_1408182086233_0013_m_000002_0 4 >> >> >> Thx >> >> [email protected]|齐忠 >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > > > -- > Marcelo -- [email protected]|齐忠 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
