Re: DataFrame groupBy MapType

2015-04-07 Thread Justin Yip
Thanks Michael. Will submit a ticket. Justin On Mon, Apr 6, 2015 at 1:53 PM, Michael Armbrust wrote: > I'll add that I don't think there is a convenient way to do this in the > Column API ATM, but would welcome a JIRA for adding it :) > > On Mon, Apr 6, 2015 at 1:45 PM, Michael Armbrust > wrot

Re: DataFrame groupBy MapType

2015-04-06 Thread Michael Armbrust
I'll add that I don't think there is a convenient way to do this in the Column API ATM, but would welcome a JIRA for adding it :) On Mon, Apr 6, 2015 at 1:45 PM, Michael Armbrust wrote: > In HiveQL, you should be able to express this as: > > SELECT ... FROM table GROUP BY m['SomeKey'] > > On Sat

Re: DataFrame groupBy MapType

2015-04-06 Thread Michael Armbrust
In HiveQL, you should be able to express this as: SELECT ... FROM table GROUP BY m['SomeKey'] On Sat, Apr 4, 2015 at 5:25 PM, Justin Yip wrote: > Hello, > > I have a case class like this: > > case class A( > m: Map[Long, Long], > ... > ) > > and constructed a DataFrame from Seq[A]. > > I wo

DataFrame groupBy MapType

2015-04-04 Thread Justin Yip
Hello, I have a case class like this: case class A( m: Map[Long, Long], ... ) and constructed a DataFrame from Seq[A]. I would like to perform a groupBy on A.m("SomeKey"). I can implement a UDF, create a new Column then invoke a groupBy on the new Column. But is it the idiomatic way of doin