RM NM logs traced below,
RM -->
2016-03-30 14:59:15,498 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher:
Setting up container Container: [ContainerId:
container_1459326455972_0004_01_000001, NodeId: myhost:60653,
NodeHttpAddress: myhost:8042, Resource: <memory:1024, vCores:1>, Priority:
0, Token: Token { kind: ContainerToken, service: 10.20.53.123:60653 }, ]
for AM appattempt_1459326455972_0004_000001
2016-03-30 14:59:15,498 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher:
Command to launch container container_1459326455972_0004_01_000001 :
{{JAVA_HOME}}/bin/java,-server,-Xmx512m,-Djava.io.tmpdir={{PWD}}/tmp,-Dspark.yarn.app.container.log.dir=<LOG_DIR>,-XX:MaxPermSize=256m,org.apache.spark.deploy.yarn.ExecutorLauncher,--arg,'
10.20.53.123:45379
',--executor-memory,1024m,--executor-cores,1,--properties-file,{{PWD}}/__spark_conf__/__spark_conf__.properties,1>,<LOG_DIR>/stdout,2>,<LOG_DIR>/stderr
2016-03-30 14:59:15,498 INFO
org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager:
Create AMRMToken for ApplicationAttempt:
appattempt_1459326455972_0004_000001
2016-03-30 14:59:15,498 INFO
org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager:
Creating password for appattempt_1459326455972_0004_000001
2016-03-30 14:59:15,533 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Done
launching container Container: [ContainerId:
container_1459326455972_0004_01_000001, NodeId: myhost:60653,
NodeHttpAddress: myhost:8042, Resource: <memory:1024, vCores:1>, Priority:
0, Token: Token { kind: ContainerToken, service: 10.20.53.123:60653 }, ]
for AM appattempt_1459326455972_0004_000001
2016-03-30 14:59:15,533 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1459326455972_0004_000001 State change from ALLOCATED to LAUNCHED
2016-03-30 14:59:16,437 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
container_1459326455972_0004_01_000001 Container Transitioned from ACQUIRED
to RUNNING
2016-03-30 14:59:28,514 INFO SecurityLogger.org.apache.hadoop.ipc.Server:
Auth successful for appattempt_1459326455972_0004_000001 (auth:SIMPLE)
2016-03-30 14:59:28,527 INFO
org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: AM
registration appattempt_1459326455972_0004_000001
2016-03-30 14:59:28,527 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=myhost
IP=10.20.53.123 OPERATION=Register App Master
TARGET=ApplicationMasterService RESULT=SUCCESS
APPID=application_1459326455972_0004
APPATTEMPTID=appattempt_1459326455972_0004_000001
2016-03-30 14:59:28,527 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1459326455972_0004_000001 State change from LAUNCHED to RUNNING
2016-03-30 14:59:28,528 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_1459326455972_0004 State change from ACCEPTED to RUNNING
2016-03-30 14:59:29,456 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
container_1459326455972_0004_01_000002 Container Transitioned from NEW to
ALLOCATED
2016-03-30 14:59:29,457 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=myhost OPERATION=AM
Allocated Container TARGET=SchedulerApp RESULT=SUCCESS
APPID=application_1459326455972_0004
CONTAINERID=container_1459326455972_0004_01_000002
2016-03-30 14:59:29,457 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode:
Assigned container container_1459326455972_0004_01_000002 of capacity
<memory:1536, vCores:1> on host myhost:60653, which has 2 containers,
<memory:2560, vCores:2> used and <memory:1468, vCores:2> available after
allocation
2016-03-30 14:59:30,121 INFO
org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM:
Sending NMToken for nodeId : myhost:60653 for container :
container_1459326455972_0004_01_000002
2016-03-30 14:59:30,122 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
container_1459326455972_0004_01_000002 Container Transitioned from
ALLOCATED to ACQUIRED
2016-03-30 14:59:30,458 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt:
Making reservation: node=myhost app_id=application_1459326455972_0004
2016-03-30 14:59:30,458 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
container_1459326455972_0004_01_000003 Container Transitioned from NEW to
RESERVED
2016-03-30 14:59:30,458 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerNode:
Reserved container container_1459326455972_0004_01_000003 on node host:
myhost:60653 #containers=2 available=1468 used=2560 for application
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt@57cf4903
2016-03-30 14:59:31,460 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
container_1459326455972_0004_01_000002 Container Transitioned from ACQUIRED
to RUNNING
NM -->
2016-03-30 15:02:38,537 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 22899 for container-id
container_1459326455972_0004_01_000002: 318.7 MB of 1.5 GB physical memory
used; 1.7 GB of 3.1 GB virtual memory used
2016-03-30 15:02:38,549 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 22813 for container-id
container_1459326455972_0004_01_000001: 172.2 MB of 1 GB physical memory
used; 1.2 GB of 2.1 GB virtual memory used
2016-03-30 15:02:41,564 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 22899 for container-id
container_1459326455972_0004_01_000002: 318.7 MB of 1.5 GB physical memory
used; 1.7 GB of 3.1 GB virtual memory used
2016-03-30 15:02:41,575 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 22813 for container-id
container_1459326455972_0004_01_000001: 172.2 MB of 1 GB physical memory
used; 1.2 GB of 2.1 GB virtual memory used
Looks like container is not getting enough resources to spawn.
On Wed, Mar 30, 2016 at 3:27 AM, Alexander Pivovarov <apivova...@gmail.com>
wrote:
> ok, start EMR-4.3.0 or 4.2.0 cluster and look at how to configure spark on
> yarn properly
>
