Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4747#discussion_r25379656
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -63,6 +63,7 @@ private[spark] object Utils extends Logging {
val random = new Random()
private val MAX_DIR_CREATION_ATTEMPTS: Int = 10
+ @volatile private var localRootDirs: Array[String] = null
--- End diff --
OK at first I was uneasy about adding more global state to a common `Utils`
class, but this reflects configuration that's already global to the whole JVM,
coming by sys properties like `SPARK_LOCAL_DIRS`. Maybe this could be declared
nearer its usage but that alone isn't worth changing this. I also don't see a
way to avoid the double-checked locking; I couldn't get something clean with
`AtomicReference` to work. I think the extra `SPARK_EXECUTOR_DIRS` is worth it;
it's not exposed otherwise.
In other words, LGTM. Any other reviews? I'll merge after waiting a short
while for comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
