Re: explain codegen

2016-04-04 Thread Ted Yu
Thanks to all who have responded. It turned out that the following command line for maven caused the error (I forgot to include this in first email): eclipse:eclipse Once I omitted the above, 'explain codegen' works. On Mon, Apr 4, 2016 at 9:37 AM, Reynold Xin wrote: > Why don't you wipe every

Re: explain codegen

2016-04-04 Thread Reynold Xin
Why don't you wipe everything out and try again? On Monday, April 4, 2016, Ted Yu wrote: > The commit you mentioned was made Friday. > I refreshed workspace Sunday - so it was included. > > Maybe this was related: > > $ bin/spark-shell > Failed to find Spark jars directory > (/home/hbase/spark/a

Re: explain codegen

2016-04-04 Thread Ted Yu
The commit you mentioned was made Friday. I refreshed workspace Sunday - so it was included. Maybe this was related: $ bin/spark-shell Failed to find Spark jars directory (/home/hbase/spark/assembly/target/scala-2.10). You need to build Spark before running this program. Then I did: $ ln -s /ho

Re: explain codegen

2016-04-04 Thread Herman van Hövell tot Westerflier
No, it can''t. You only need implicits when you are using the catalyst DSL. The error you get is due to the fact that the parser does not recognize the CODEGEN keyword (which was the case before we introduced this in https://github.com/apache/spark/commit/fa1af0aff7bde9bbf7bfa6a3ac74699734c2fd8a).

Re: explain codegen

2016-04-04 Thread Ted Yu
Could the error I encountered be due to missing import(s) of implicit ? Thanks On Sun, Apr 3, 2016 at 9:42 PM, Reynold Xin wrote: > Works for me on latest master. > > > > scala> sql("explain codegen select 'a' as a group by 1").head > res3: org.apache.spark.sql.Row = > [Found 2 WholeStageCodege

Re: explain codegen

2016-04-03 Thread Reynold Xin
Works for me on latest master. scala> sql("explain codegen select 'a' as a group by 1").head res3: org.apache.spark.sql.Row = [Found 2 WholeStageCodegen subtrees. == Subtree 1 / 2 == WholeStageCodegen : +- TungstenAggregate(key=[], functions=[], output=[a#10]) : +- INPUT +- Exchange SingleP

Re: explain codegen

2016-04-03 Thread Jacek Laskowski
Hi, Looks related to the recent commit... Repository: spark Updated Branches: refs/heads/master 2262a9335 -> 1f0c5dceb [SPARK-14350][SQL] EXPLAIN output should be in a single cell Jacek 03.04.2016 7:00 PM "Ted Yu" napisał(a): > Hi, > Based on master branch refreshed today, I issued 'git cle