Hi Hao,
Thanks. I'll take a look at it.
On Wednesday, December 30, 2015 12:47 PM, "Cheng, Hao"
<hao.ch...@intel.com> wrote:
#yiv7928789615 #yiv7928789615 -- _filtered #yiv7928789615
{font-family:Helvetica;panose-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv7928789615
{font-family:宋体;panose-1:2 1 6 0 3 1 1 1 1 1;} _filtered #yiv7928789615
{panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv7928789615
{font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv7928789615
{font-family:Verdana;panose-1:2 11 6 4 3 5 4 4 2 4;} _filtered #yiv7928789615
{panose-1:2 1 6 0 3 1 1 1 1 1;}#yiv7928789615 #yiv7928789615
p.yiv7928789615MsoNormal, #yiv7928789615 li.yiv7928789615MsoNormal,
#yiv7928789615 div.yiv7928789615MsoNormal
{margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;}#yiv7928789615 a:link,
#yiv7928789615 span.yiv7928789615MsoHyperlink
{color:#0563C1;text-decoration:underline;}#yiv7928789615 a:visited,
#yiv7928789615 span.yiv7928789615MsoHyperlinkFollowed
{color:#954F72;text-decoration:underline;}#yiv7928789615
span.yiv7928789615EmailStyle17 {color:#1F497D;}#yiv7928789615
.yiv7928789615MsoChpDefault {font-size:10.0pt;} _filtered #yiv7928789615
{margin:72.0pt 90.0pt 72.0pt 90.0pt;}#yiv7928789615
div.yiv7928789615WordSection1 {}#yiv7928789615 Hi, currently, the Simple SQL
Parser of SQLContext is quite weak, and doesn’t support the rollup, but you can
check the code https://github.com/apache/spark/pull/5080/ , which aimed to
add the support, just in case you can patch it in your own branch. In Spark
2.0, the simple SQL Parser will be replaced by HQL Parser, so it will not be
the problem then. Hao From: Yi Zhang [mailto:zhangy...@yahoo.com.INVALID]
Sent: Wednesday, December 30, 2015 11:41 AM
To: User
Subject: Does Spark SQL support rollup like HQL Hi guys, As we know,
hqlContext support rollup like this: hiveContext.sql("select a, b, sum(c)
from t group by a, b with rollup") And I also knows that dataframe provides
rollup function to support it: dataframe.rollup($"a", $"b").agg(Map("c" ->
"sum")) But in my scenario, I'd better use sql syntax in SqlContext to
support rollup it seems like what HqlContext does. Any suggestion? Thanks.
Regards, Yi Zhang
