Hi, currently, the Simple SQL Parser of SQLContext is quite weak, and doesn’t support the rollup, but you can check the code
https://github.com/apache/spark/pull/5080/ , which aimed to add the support, just in case you can patch it in your own branch. In Spark 2.0, the simple SQL Parser will be replaced by HQL Parser, so it will not be the problem then. Hao From: Yi Zhang [mailto:zhangy...@yahoo.com.INVALID] Sent: Wednesday, December 30, 2015 11:41 AM To: User Subject: Does Spark SQL support rollup like HQL Hi guys, As we know, hqlContext support rollup like this: hiveContext.sql("select a, b, sum(c) from t group by a, b with rollup") And I also knows that dataframe provides rollup function to support it: dataframe.rollup($"a", $"b").agg(Map("c" -> "sum")) But in my scenario, I'd better use sql syntax in SqlContext to support rollup it seems like what HqlContext does. Any suggestion? Thanks. Regards, Yi Zhang
