Interesting, I believe we have run that query with version 1.1.0 with codegen turned on and not much has changed there. Is the error deterministic?
On Mon, Nov 17, 2014 at 7:04 PM, Eric Zhen <zhpeng...@gmail.com> wrote: > Hi Michael, > > We use Spark v1.1.1-rc1 with jdk 1.7.0_51 and scala 2.10.4. > > On Tue, Nov 18, 2014 at 7:09 AM, Michael Armbrust <mich...@databricks.com> > wrote: > >> What version of Spark SQL? >> >> On Sat, Nov 15, 2014 at 10:25 PM, Eric Zhen <zhpeng...@gmail.com> wrote: >> >>> Hi all, >>> >>> We run SparkSQL on TPCDS benchmark Q19 with spark.sql.codegen=true, we >>> got exceptions as below, has anyone else saw these before? >>> >>> java.lang.ExceptionInInitializerError >>> at >>> org.apache.spark.sql.execution.SparkPlan.newProjection(SparkPlan.scala:92) >>> at >>> org.apache.spark.sql.execution.Exchange$$anonfun$execute$1$$anonfun$1.apply(Exchange.scala:51) >>> at >>> org.apache.spark.sql.execution.Exchange$$anonfun$execute$1$$anonfun$1.apply(Exchange.scala:48) >>> at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:596) >>> at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:596) >>> at >>> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) >>> at >>> org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262) >>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:229) >>> at >>> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68) >>> at >>> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) >>> at org.apache.spark.scheduler.Task.run(Task.scala:54) >>> at >>> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:178) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:744) >>> Caused by: java.lang.NullPointerException >>> at >>> scala.reflect.internal.Types$TypeRef.computeHashCode(Types.scala:2358) >>> at >>> scala.reflect.internal.Types$UniqueType.<init>(Types.scala:1304) >>> at scala.reflect.internal.Types$TypeRef.<init>(Types.scala:2341) >>> at >>> scala.reflect.internal.Types$NoArgsTypeRef.<init>(Types.scala:2137) >>> at >>> scala.reflect.internal.Types$TypeRef$$anon$6.<init>(Types.scala:2544) >>> at scala.reflect.internal.Types$TypeRef$.apply(Types.scala:2544) >>> at scala.reflect.internal.Types$class.typeRef(Types.scala:3615) >>> at >>> scala.reflect.internal.SymbolTable.typeRef(SymbolTable.scala:13) >>> at >>> scala.reflect.internal.Symbols$TypeSymbol.newTypeRef(Symbols.scala:2752) >>> at >>> scala.reflect.internal.Symbols$TypeSymbol.typeConstructor(Symbols.scala:2806) >>> at >>> scala.reflect.internal.Symbols$SymbolContextApiImpl.toTypeConstructor(Symbols.scala:103) >>> at >>> scala.reflect.internal.Symbols$TypeSymbol.toTypeConstructor(Symbols.scala:2698) >>> at >>> org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$typecreator1$1.apply(CodeGenerator.scala:46) >>> at >>> scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe$lzycompute(TypeTags.scala:231) >>> at >>> scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe(TypeTags.scala:231) >>> at scala.reflect.api.TypeTags$class.typeOf(TypeTags.scala:335) >>> at scala.reflect.api.Universe.typeOf(Universe.scala:59) >>> at >>> org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator.<init>(CodeGenerator.scala:46) >>> at >>> org.apache.spark.sql.catalyst.expressions.codegen.GenerateProjection$.<init>(GenerateProjection.scala:29) >>> at >>> org.apache.spark.sql.catalyst.expressions.codegen.GenerateProjection$.<clinit>(GenerateProjection.scala) >>> ... 15 more >>> -- >>> Best Regards >>> >> >> > > > -- > Best Regards >
