Hi Michael, Is the in-memory columnar store planned as part of SparkSQL ?
Also will both HiveQL & SQLParser be kept updated? Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Sun, May 25, 2014 at 2:44 AM, Michael Armbrust <mich...@databricks.com>wrote: > > But going back to your presented pattern, I have a question. Say your data >> does have a fixed structure, but some of the JSON values are lists. How >> would you map that to a SchemaRDD? (I didn’t notice any list values in the >> CandyCrush example.) Take the likes field from my original example: >> > Spark SQL supports complex types, including sequences. You should be able > to define a field of type Seq[String] in your case class. > > One note: the hql parser that is including in a hive context currently has > better support for working with complex types, but we are working to > improve that. >
