Re: ORC Transaction Table - Spark

2017-08-24 Thread Gopal Vijayaraghavan
> Or, is this an artifact of an incompatibility between ORC files written by > the Hive 2.x ORC serde not being readable by the Hive 1.x ORC serde? > 3. Is there a difference in the ORC file format spec. at play here? Nope, we're still defaulting to hive-0.12 format ORC files in Hive-2.x. We h

RE: ORC Transaction Table - Spark

2017-08-24 Thread Larson, Kurt
serde? 6. Any similar issues with Parquet format in Hive 1.x and 2.x? From: Aviral Agarwal [mailto:aviral12...@gmail.com] Sent: Wednesday, August 23, 2017 10:34 PM To: user@hive.apache.org Subject: Re: ORC Transaction Table - Spark So, there is no way possible right now for Spark to read

Re: ORC Transaction Table - Spark

2017-08-24 Thread Furcy Pin
r@hive.apache.org" >> *Date: *Wednesday, August 23, 2017 at 12:24 AM >> *To: *"user@hive.apache.org" >> *Subject: *Re: ORC Transaction Table - Spark >> >> >> >> Hi, >> >> Yes it caused by wrong naming convention of the delta direct

Re: ORC Transaction Table - Spark

2017-08-23 Thread Aviral Agarwal
ral Agarwal > *Reply-To: *"user@hive.apache.org" > *Date: *Wednesday, August 23, 2017 at 12:24 AM > *To: *"user@hive.apache.org" > *Subject: *Re: ORC Transaction Table - Spark > > > > Hi, > > Yes it caused by wrong naming convention of the delt

Re: ORC Transaction Table - Spark

2017-08-23 Thread Eugene Koifman
This looks like you have some data written by Hive 2.x and Hive 1.x code trying to read it. That is not supported. From: Aviral Agarwal Reply-To: "user@hive.apache.org" Date: Wednesday, August 23, 2017 at 12:24 AM To: "user@hive.apache.org" Subject: Re: ORC Transaction Tab

Re: ORC Transaction Table - Spark

2017-08-23 Thread Aviral Agarwal
2017 at 5:39 AM > *To: *"user@hive.apache.org" > *Subject: *ORC Transaction Table - Spark > > > > Hi, > > > > I am trying to read hive orc transaction table through Spark but I am > getting the following error > > > Caused by: java.

Re: ORC Transaction Table - Spark

2017-08-22 Thread Eugene Koifman
hive.apache.org" Subject: ORC Transaction Table - Spark Hi, I am trying to read hive orc transaction table through Spark but I am getting the following error Caused by: java.lang.RuntimeException: serious problem at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInput

ORC Transaction Table - Spark

2017-08-22 Thread Aviral Agarwal
Hi, I am trying to read hive orc transaction table through Spark but I am getting the following error Caused by: java.lang.RuntimeException: serious problem at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1021) at org.apache.hadoop.hive.ql.io.orc.OrcInput