Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2250#issuecomment-55798065
This seems like a good idea; I can see how the current behavior is
confusing, especially since I think it might be common for multiple apps to be
running with the same name (e.g. two copies of `spark-shell`). Do you know if
there are any other places where we incorrectly treat application names as
unique?
I'm not sure that calling `System.currentTimeMillis()` is the most
intuitive way to give applications unique ids, though. The Spark master
already gives application unique ids, such as `app-20140916115848-0000`, and it
would be nice if we used these same IDs in the metrics system.
The Master-assigned application ID is exposed through
`TaskScheduler.applicationId()` and comes from
`SchedulerBackend.applicationId()`; technically this method could return `None`
but it looks like none of the current implementations do. (This id was exposed
pretty recently; see #1218). This id is only available after the application
is registered with the master, but that shouldn't be a problem since
`initDriverMetrics()` is called after the task scheduler has been initialized.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
