Re: flattening a list in spark sql

2014-09-10 Thread gtinside
My bad, please ignore, it works !!! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/flattening-a-list-in-spark-sql-tp13300p13901.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --

Re: flattening a list in spark sql

2014-09-10 Thread gtinside
Hi , Thanks it worked, really appreciate your help. I have also been trying to do multiple Lateral Views, but it doesn't seem to be working. Query : hiveContext.sql("Select t2 from fav LATERAL VIEW explode(TABS) tabs1 as t1 LATERAL VIEW explode(t1) tabs2 as t2") Exception org.apache.spark.sql.c

Re: flattening a list in spark sql

2014-09-02 Thread Michael Armbrust
Yes you can. HiveContext's functionality is a strict superset of SQLContext. On Tue, Sep 2, 2014 at 6:35 PM, gtinside wrote: > Thanks . I am not using hive context . I am loading data from Cassandra and > then converting it into json and then querying it through SQL context . Can > I use use h

Re: flattening a list in spark sql

2014-09-02 Thread gtinside
Thanks . I am not using hive context . I am loading data from Cassandra and then converting it into json and then querying it through SQL context . Can I use use hive context to query on a jsonRDD ? Gaurav -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/fla

Re: flattening a list in spark sql

2014-09-02 Thread Michael Armbrust
Check out LATERAL VIEW explode: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+LateralView On Tue, Sep 2, 2014 at 1:26 PM, gtinside wrote: > Hi , > > I am using jsonRDD in spark sql and having trouble iterating through array > inside the json object. Please refer to the schema