Hello:
I'm using the history server to keep track of the applications I run in my
cluster. I'm using Spark with YARN.
When I run on application it finishes correctly even YARN says that it
finished. This is the result of the YARN Resource Manager API
{u'app': [{u'runningContainers': -1, u'allocatedVCores': -1, u'clusterId':
1460540049690, u'amContainerLogs':
u'http://parapide-2.rennes.grid5000.fr:8042/node/containerlogs/container_1460540049690_0001_01_000001/abrandon',
u'id': u'*application_1460540049690_0001*', u'preemptedResourceMB': 0,
u'finishedTime': 1460550170085, u'numAMContainerPreempted': 0, u'user':
u'abrandon', u'preemptedResourceVCores': 0, u'startedTime': 1460548211207,
u'elapsedTime': 1958878, u'state': u'FINISHED',
u'numNonAMContainerPreempted': 0, u'progress': 100.0, u'trackingUI':
u'History', u'trackingUrl':
u'http://paranoia-1.rennes.grid5000.fr:8088/proxy/application_1460540049690_0001/A',
u'allocatedMB': -1, u'amHostHttpAddress':
u'parapide-2.rennes.grid5000.fr:8042', u'memorySeconds': 37936274,
u'applicationTags': u'', u'name': u'KMeans', u'queue': u'default',
u'vcoreSeconds': 13651, u'applicationType': u'SPARK', u'diagnostics': u'',
u'finalStatus': u'*SUCCEEDED*'}
However when I query the SPARK UI
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n26777/Screen_Shot_2016-04-13_at_14.png>
You can see that for Job ID 2 no tasks have run and I can't get information
about them. Is this some kind of bug?
Thanks for your help as always
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Problem-with-History-Server-tp26777.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
