Re: [hive context] Unable to query array once saved as parquet

/gist.github.com/ayoub-benali/54d6f3b8635530e4e936> >> >> Could the problem comes from the way I insert the data into the table ? >> >> Is this problem related to this JIRA ticket >> https://issues.apache.org/jira/browse/SPARK-5236 ? >> >> Because I got a similar exception "GenericRow cannot be cast to >> org.apache.spark.sql.catalyst.expressions.SpecificMutableRow" With an other >> table that contains also a array of struct. >> >> Thanks, >> Ayoub. >> >> -- >> View this message in context: [hive context] Unable to query array once >> saved as parquet >> <http://apache-spark-user-list.1001560.n3.nabble.com/hive-context-Unable-to-query-array-once-saved-as-parquet-tp21446.html> >> Sent from the Apache Spark User List mailing list archive >> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >> > > > -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Re-hive-context-Unable-to-query-array-once-saved-as-parquet-tp21628.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: [hive context] Unable to query array once saved as parquet

.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) >>> at >>> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) >>> at >>> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) >>> >>> The full code leading to this issue is available here: gist >>> <https://gist.github.com/ayoub-benali/54d6f3b8635530e4e936> >>> >>> Could the problem comes from the way I insert the data into the table ? >>> >>> Is this problem related to this JIRA ticket >>> https://issues.apache.org/jira/browse/SPARK-5236 ? >>> >>> Because I got a similar exception "GenericRow cannot be cast to >>> org.apache.spark.sql.catalyst.expressions.SpecificMutableRow" With an other >>> table that contains also a array of struct. >>> >>> Thanks, >>> Ayoub. >>> >>> -- >>> View this message in context: [hive context] Unable to query array once >>> saved as parquet >>> <http://apache-spark-user-list.1001560.n3.nabble.com/hive-context-Unable-to-query-array-once-saved-as-parquet-tp21446.html> >>> Sent from the Apache Spark User List mailing list archive >>> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >>> >> >> >> > -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Re-hive-context-Unable-to-query-array-once-saved-as-parquet-tp21466.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: [hive context] Unable to query array once saved as parquet

t; at >> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) >> >> The full code leading to this issue is available here: gist >> <https://gist.github.com/ayoub-benali/54d6f3b8635530e4e936> >> >> Could the problem comes from

Re: [hive context] Unable to query array once saved as parquet

According to the Gist Ayoub provided, the schema is fine. I reproduced this issue locally, it should be bug, but I don't think it's related to SPARK-5236. Will investigate this soon. Ayoub - would you mind to help to file a JIRA for this issue? Thanks! Cheng On 1/30/15 11:28 AM, Michael Armbr

Re: [hive context] Unable to query array once saved as parquet

According to the Gist Ayoub provided, the schema is fine. I reproduced this issue locally, it should be bug, but I don't think it's related to SPARK-5236. Will investigate this soon. Ayoub - would you mind to help to file a JIRA for this issue? Thanks! Cheng On 1/30/15 11:28 AM, Michael Armbr

Re: [hive context] Unable to query array once saved as parquet

/ayoub-benali/54d6f3b8635530e4e936> >> >> Could the problem comes from the way I insert the data into the table ? >> >> Is this problem related to this JIRA ticket >> https://issues.apache.org/jira/browse/SPARK-5236 ? >> >> Because I got a similar exception "GenericRow cannot be cast to >> org.apache.spark.sql.catalyst.expressions.SpecificMutableRow" With an other >> table that contains also a array of struct. >> >> Thanks, >> Ayoub. >> >> -- >> View this message in context: [hive context] Unable to query array once >> saved as parquet >> <http://apache-spark-user-list.1001560.n3.nabble.com/hive-context-Unable-to-query-array-once-saved-as-parquet-tp21446.html> >> Sent from the Apache Spark User List mailing list archive >> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >> > > -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Re-hive-context-Unable-to-query-array-once-saved-as-parquet-tp21448.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: [hive context] Unable to query array once saved as parquet

Is it possible that your schema contains duplicate columns or column with spaces in the name? The parquet library will often give confusing error messages in this case. On Fri, Jan 30, 2015 at 10:33 AM, Ayoub wrote: > Hello, > > I have a problem when querying, with a hive context on spark > 1.2