Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/3082#discussion_r19792416
--- Diff:
yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala
---
@@ -90,6 +90,12 @@ class ExecutorRunnable(
ctx.setApplicationACLs(YarnSparkHadoopUtil.getApplicationAclsForYarn(securityMgr))
+ // If external shuffle service is enabled, register with the
+ // Yarn shuffle service already started on the node manager
+ if (sparkConf.getBoolean("spark.shuffle.service.enabled", false)) {
+ ctx.setServiceData(Map[String, ByteBuffer]("spark_shuffle" ->
ByteBuffer.allocate(0)))
--- End diff --
Apparently if this map is empty, then YARN won't ever register with the
auxiliary service. So I believe this is a place holder to force registration.
However now that I think about it @andrewor14 - can we just go ahead and pass
the secret key here? We should have it since we are inside of the executor
runner.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
