Is there some place I can read more about it ? I can't find any reference.
I actully want to flatten these structures and not return them from the UDF.
Thanks,
Daniel
On Tue, Nov 25, 2014 at 8:44 PM, Michael Armbrust
wrote:
> Maps should just be scala maps, structs are rows inside of rows. If
Maps should just be scala maps, structs are rows inside of rows. If you
wan to return a struct from a UDF you can do that with a case class.
On Tue, Nov 25, 2014 at 10:25 AM, Daniel Haviv
wrote:
> Thank you.
>
> How can I address more complex columns like maps and structs?
>
> Thanks again!
> D
Thank you.
How can I address more complex columns like maps and structs?
Thanks again!
Daniel
> On 25 בנוב׳ 2014, at 19:43, Michael Armbrust wrote:
>
> Probably the easiest/closest way to do this would be with a UDF, something
> like:
>
> registerFunction("makeString", (s: Seq[String]) => s.
Probably the easiest/closest way to do this would be with a UDF, something
like:
registerFunction("makeString", (s: Seq[String]) => s.mkString(","))
sql("SELECT *, makeString(c8) AS newC8 FROM jRequests")
Although this does not modify a column, but instead appends a new column.
Another more comp
Hi,
I'm selecting columns from a json file, transform some of them and would
like to store the result as a parquet file but I'm failing.
This is what I'm doing:
val jsonFiles=sqlContext.jsonFile("/requests.loading")
jsonFiles.registerTempTable("jRequests")
val clean_jRequests=sqlContext.sql("sel
