Spark yarn executor container fails if yarn.nodemanager.local-dirs starts
with file://

   <property>
     <name>yarn.nodemanager.local-dirs</name>
     <value>file:///data01/yarn/nm,file:///data02/yarn/nm</value>
   </property>

other application, e.g. Hadoop MR and Hive work normally

Spark works only if yarn.nodemanager.local-dirs does not have file:// prefix
e.g.
    <value>/data01/yarn/nm,/data02/yarn/nm</value>

https://issues.apache.org/jira/browse/SPARK-13532

Reply via email to