This is actually a problem with our use of Scala's reflection library. Unfortunately you need to load Spark SQL using the primordial classloader, otherwise you run into this problem. If anyone from the scala side can hint how we can tell scala.reflect which classloader to use when creating the mirror I'd love to fix this problem.
On Thu, Apr 2, 2015 at 6:00 AM, Dean Wampler <deanwamp...@gmail.com> wrote: > It failed to find the class class > org.apache.spark.sql.catalyst.ScalaReflection > in the Spark SQL library. Make sure it's in the classpath and the version > is correct, too. > > Dean Wampler, Ph.D. > Author: Programming Scala, 2nd Edition > <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) > Typesafe <http://typesafe.com> > @deanwampler <http://twitter.com/deanwampler> > http://polyglotprogramming.com > > On Thu, Apr 2, 2015 at 8:39 AM, Sathish Kumaran Vairavelu < > vsathishkuma...@gmail.com> wrote: > >> Hi Everyone, >> >> I am getting following error while registering table using Scala IDE. >> Please let me know how to resolve this error. I am using Spark 1.2.1 >> >> import sqlContext.createSchemaRDD >> >> >> >> val empFile = sc.textFile("/tmp/emp.csv", 4) >> >> .map ( _.split(",") ) >> >> .map( row=> Employee(row(0),row(1), row(2), row(3), >> row(4))) >> >> empFile.registerTempTable("Employees") >> >> Thanks >> >> Sathish >> >> Exception in thread "main" >> scala.reflect.internal.MissingRequirementError: class >> org.apache.spark.sql.catalyst.ScalaReflection in JavaMirror with primordial >> classloader with boot classpath >> [/Applications/eclipse/plugins/org.scala-ide.scala210.jars_4.0.0.201412161056/target/jars/scala-library.jar:/Applications/eclipse/plugins/org.scala-ide.scala210.jars_4.0.0.201412161056/target/jars/scala-reflect.jar:/Applications/eclipse/plugins/org.scala-ide.scala210.jars_4.0.0.201412161056/target/jars/scala-actor.jar:/Applications/eclipse/plugins/org.scala-ide.scala210.jars_4.0.0.201412161056/target/jars/scala-swing.jar:/Applications/eclipse/plugins/org.scala-ide.scala210.jars_4.0.0.201412161056/target/jars/scala-compiler.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_65.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_65.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_65.jdk/Contents/Home/jre/lib/sunrsasign.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_65.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_65.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_65.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_65.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_65.jdk/Contents/Home/jre/classes] >> not found. >> >> at scala.reflect.internal.MissingRequirementError$.signal( >> MissingRequirementError.scala:16) >> >> at scala.reflect.internal.MissingRequirementError$.notFound( >> MissingRequirementError.scala:17) >> >> at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass( >> Mirrors.scala:48) >> >> at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass( >> Mirrors.scala:61) >> >> at scala.reflect.internal.Mirrors$RootsBase.staticModuleOrClass( >> Mirrors.scala:72) >> >> at scala.reflect.internal.Mirrors$RootsBase.staticClass(Mirrors.scala:119 >> ) >> >> at scala.reflect.internal.Mirrors$RootsBase.staticClass(Mirrors.scala:21) >> >> at org.apache.spark.sql.catalyst.ScalaReflection$$typecreator1$1.apply( >> ScalaReflection.scala:115) >> >> at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe$lzycompute( >> TypeTags.scala:231) >> >> at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe(TypeTags.scala:231) >> >> at scala.reflect.api.TypeTags$class.typeOf(TypeTags.scala:335) >> >> at scala.reflect.api.Universe.typeOf(Universe.scala:59) >> >> at org.apache.spark.sql.catalyst.ScalaReflection$class.schemaFor( >> ScalaReflection.scala:115) >> >> at org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor( >> ScalaReflection.scala:33) >> >> at org.apache.spark.sql.catalyst.ScalaReflection$class.schemaFor( >> ScalaReflection.scala:100) >> >> at org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor( >> ScalaReflection.scala:33) >> >> at org.apache.spark.sql.catalyst.ScalaReflection$class.attributesFor( >> ScalaReflection.scala:94) >> >> at org.apache.spark.sql.catalyst.ScalaReflection$.attributesFor( >> ScalaReflection.scala:33) >> >> at org.apache.spark.sql.SQLContext.createSchemaRDD(SQLContext.scala:111) >> >> at com.svairavelu.examples.QueryCSV$.main(QueryCSV.scala:24) >> >> at com.svairavelu.examples.QueryCSV.main(QueryCSV.scala) >> > >
