Just sent out a PR[1] to support cube/rollup as function, it works with both SQLContext and HiveContext.
https://github.com/apache/spark/pull/10522/files On Tue, Dec 29, 2015 at 9:35 PM, Yi Zhang <zhangy...@yahoo.com.invalid> wrote: > Hi Hao, > > Thanks. I'll take a look at it. > > > On Wednesday, December 30, 2015 12:47 PM, "Cheng, Hao" <hao.ch...@intel.com> > wrote: > > > Hi, currently, the Simple SQL Parser of SQLContext is quite weak, and > doesn’t support the rollup, but you can check the code > > https://github.com/apache/spark/pull/5080/ , which aimed to add the support, > just in case you can patch it in your own branch. > > In Spark 2.0, the simple SQL Parser will be replaced by HQL Parser, so it > will not be the problem then. > > Hao > > From: Yi Zhang [mailto:zhangy...@yahoo.com.INVALID] > Sent: Wednesday, December 30, 2015 11:41 AM > To: User > Subject: Does Spark SQL support rollup like HQL > > Hi guys, > > As we know, hqlContext support rollup like this: > > hiveContext.sql("select a, b, sum(c) from t group by a, b with rollup") > > And I also knows that dataframe provides rollup function to support it: > > dataframe.rollup($"a", $"b").agg(Map("c" -> "sum")) > > But in my scenario, I'd better use sql syntax in SqlContext to support > rollup it seems like what HqlContext does. Any suggestion? > > Thanks. > > Regards, > Yi Zhang > > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
