While running a my Spark Application over 1.3.0 with Scala 2.10.0 i
encountered

15/04/21 09:13:21 ERROR executor.Executor: Exception in task 7.0 in stage
2.0 (TID 28)
java.lang.UnsupportedOperationException: tail of empty list
at scala.collection.immutable.Nil$.tail(List.scala:339)
at scala.collection.immutable.Nil$.tail(List.scala:334)
at scala.reflect.internal.SymbolTable.popPhase(SymbolTable.scala:172)


My app has quite a lot of  reflection usage as i need to build avro schema.
I was told on stack overflow that this error is because reflection is not
thread safe in scala 2.10 and was advised to use scala 2.11

Hence I moved to scala 2.11 and moved all spark maven dependencies to 2.11
(spark-avro is still 2.10 as 2.11 is not available). I ran into this

15/04/21 09:13:21 ERROR executor.Executor: Exception in task 7.0 in stage
2.0 (TID 28)
java.lang.UnsupportedOperationException: tail of empty list
at scala.collection.immutable.Nil$.tail(List.scala:339)
at scala.collection.immutable.Nil$.tail(List.scala:334)
at scala.reflect.internal.SymbolTable.popPhase(SymbolTable.scala:172)


Code:
    args.map {
      option =>
        {
          val pairs = option.split("=")
          arguments += pairs(0) -> pairs(1)
        }
    }

Error is at ->. Does Spark use scala libs 2.10 at runtime ? Any suggestions
for fix ?
-- 
Deepak

Reply via email to