Re: ORC ACID table returning Array Index Out of Bounds

2018-02-15 Thread Aviral Agarwal
1]; } if (!isTail) { maxKey = keyIndex[firstStripe + stripeCount - 1]; } } If this is still an open issue I would like to submit a patch to it. Let me know how can I further debug this issue. Thanks, Aviral Agarwal On Feb 15, 2018 23:10, "Eugene Koifman" wrote: > What versio

ORC ACID table returning Array Index Out of Bounds

2018-02-15 Thread Aviral Agarwal
) ... 26 more Any help would be appreciated. Regards, Aviral Agarwal

ORC Schema evolution - Drop columns

2017-09-04 Thread Aviral Agarwal
Hi, Is there any JIRA tracking the feature for Dropping columns from ORC transaction tables ? Regards, Aviral Agarwal

Re: ORC Transaction Table - Spark

2017-08-23 Thread Aviral Agarwal
So, there is no way possible right now for Spark to read Hive 2.x data ? On Thu, Aug 24, 2017 at 12:17 AM, Eugene Koifman wrote: > This looks like you have some data written by Hive 2.x and Hive 1.x code > trying to read it. > > That is not supported. > > > > *From: *Avi

Re: ORC Transaction Table - Spark

2017-08-23 Thread Aviral Agarwal
Hi, Yes it caused by wrong naming convention of the delta directory : /apps/hive/warehouse/foo.db/bar/year=2017/month=5/delta_0645253_0645253_0001 How do I solve this ? Thanks ! Aviral Agarwal On Tue, Aug 22, 2017 at 11:50 PM, Eugene Koifman wrote: > Could you do recursive “ls” in y

ORC Transaction Table - Spark

2017-08-22 Thread Aviral Agarwal
k.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:998) ... 118 more Any help would be appreciated. Thanks and Regards, Aviral Agarwal

Re: Hive Merge Fails with bucketid = -1

2017-07-17 Thread Aviral Agarwal
I found the solution to this. It was because the partition I was merging to had data from external source. Regards, Aviral Agarwal On Mon, Jul 17, 2017 at 4:13 PM, Aviral Agarwal wrote: > Hi , > > I am getting the following error when using Hive Merge > > Vertex failed, verte

Hive Merge Fails with bucketid = -1

2017-07-17 Thread Aviral Agarwal
a:274) ... 16 more Caused by: java.lang.ArrayIndexOutOfBoundsException: -1 at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:790) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:841) at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88) at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343) ... 17 more Any help would be appreciated. Thanks, Aviral Agarwal Date Engineer PhonePe

Re: Re:Re: Re:Re:Re:Re: How to access linux kerberosed hive from windows eclipse workspace?

2016-07-05 Thread Aviral Agarwal
n(KerberosTest.java:50) >> >Caused by: org.apache.thrift.transport.TTransportException: GSS initiate >> failed >> > at >> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) >> > at >> org.apache.thrift.t

Re: Re:Re:Re:Re: How to access linux kerberosed hive from windows eclipse workspace?

2016-07-04 Thread Aviral Agarwal
Hi, Could you enable kerberos logs with -Dsun.security.krb5.debug=true and paste the output ? On Mon, Jul 4, 2016 at 3:47 PM, Maria wrote: > The qestion "kinit: Ticket expired while renewing credentials" has > been solved. I can successfully execute "kinit -R", > but the error “java.lang.Runtim

Re: Optimized Hive query

2016-06-15 Thread Aviral Agarwal
t c from d Thanks, Aviral Agarwal On Wed, Jun 15, 2016 at 6:24 AM, Gopal Vijayaraghavan wrote: > > > So I was hoping of using internal Hive CBO to somehow change the AST > >generated for the query somehow. > > Hive does have an "explain rewrite" but that prints out

Re: Optimized Hive query

2016-06-14 Thread Aviral Agarwal
Hi, Thanks for the replies. I already knew that the optimizer already does that. My usecase is a bit different though. I want to display the flattened query back to the user. So I was hoping of using internal Hive CBO to somehow change the AST generated for the query somehow. Thanks, Aviral On Tu

Re: Optimized Hive query

2016-06-13 Thread Aviral Agarwal
Yes I want to flatten the query. Also the Insert code is correct. Thanks, Aviral Agarwal On Tue, Jun 14, 2016 at 3:46 AM, Mich Talebzadeh wrote: > you want to flatten the query I understand. > > create temporary table tmp as select c from d; > > INSERT INTO TABLE a > SELEC

Optimized Hive query

2016-06-13 Thread Aviral Agarwal
nesting levels present. Thanks, Aviral Agarwal