Without the rest of your code it's hard to make sense of errors. Why do you need to use reflection?
Make sure you use the same Scala versions throughout and 2.10.4 is recommended. That's still the official version for Spark, even though provisional support for 2.11 exists. Dean Wampler, Ph.D. Author: Programming Scala, 2nd Edition <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) Typesafe <http://typesafe.com> @deanwampler <http://twitter.com/deanwampler> http://polyglotprogramming.com On Tue, Apr 21, 2015 at 12:26 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepuj...@gmail.com> wrote: > While running a my Spark Application over 1.3.0 with Scala 2.10.0 i > encountered > > 15/04/21 09:13:21 ERROR executor.Executor: Exception in task 7.0 in stage > 2.0 (TID 28) > java.lang.UnsupportedOperationException: tail of empty list > at scala.collection.immutable.Nil$.tail(List.scala:339) > at scala.collection.immutable.Nil$.tail(List.scala:334) > at scala.reflect.internal.SymbolTable.popPhase(SymbolTable.scala:172) > > > My app has quite a lot of reflection usage as i need to build avro > schema. I was told on stack overflow that this error is because reflection > is not thread safe in scala 2.10 and was advised to use scala 2.11 > > Hence I moved to scala 2.11 and moved all spark maven dependencies to 2.11 > (spark-avro is still 2.10 as 2.11 is not available). I ran into this > > 15/04/21 09:13:21 ERROR executor.Executor: Exception in task 7.0 in stage > 2.0 (TID 28) > java.lang.UnsupportedOperationException: tail of empty list > at scala.collection.immutable.Nil$.tail(List.scala:339) > at scala.collection.immutable.Nil$.tail(List.scala:334) > at scala.reflect.internal.SymbolTable.popPhase(SymbolTable.scala:172) > > > Code: > args.map { > option => > { > val pairs = option.split("=") > arguments += pairs(0) -> pairs(1) > } > } > > Error is at ->. Does Spark use scala libs 2.10 at runtime ? Any > suggestions for fix ? > -- > Deepak > >
