Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/586#issuecomment-41699497
@sryza @tgravescs
Tom - if `spark.authenticate` is true, then is it not getting set directly
when the executor is launched:
https://github.com/apache/spark/blob/master/yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnableUtil.scala#L55
? Is there an ordering issue here, or maybe just a bug in the way it's
propagated?
Right now there isn't any SparkConf equivalent to SPARK_YARN_USER_ENV, so I
think we can just leave it for now. The other deployment modes don't currently
allow setting arbitrary env vars. Maybe in the future we can generalize it to a
conf option (and of course backwards-support), but I'm fine to leave it as-is
for now.
Re: quotes ah I guess the quoted stuff didn't work in 0.9.0 either. We
should try to fix it for 1.1 but seems fine to leave it out in that case. We
could consider using internal env vars for this when we transfer them, rather
than setting system properties in JAVA_OPTS. Not totally sure what's best.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
