HIVE:1.2, Query taking huge time

2015-08-20 Thread Nishant Aggarwal
t1.col16)) as AVG5 from huge_numeric_table_orc2 t1 left outer join huge_numeric_table_orc2 t2 on t1.col1=t2.col1 where (t1.col1) > 34.11 and (t2.col1) >10.12 *The problem is that this query gets stuck at reducers :80-85%. and goes in a loop and never finishes. * Version of Hive is 1.2. Please help. Thanks

Re: HIVE:1.2, Query taking huge time

2015-08-20 Thread Nishant Aggarwal
Thanks for the reply Noam. I have already tried the later point of dividing the query. But the challenge comes during the joining of the table. Thanks and Regards Nishant Aggarwal, PMP Cell No:- +91 99588 94305 On Thu, Aug 20, 2015 at 2:19 PM, Noam Hasson wrote: > Hi, > > Have yo

Re: HIVE:1.2, Query taking huge time

2015-08-21 Thread Nishant Aggarwal
Thanks All. I will implement the suggested points and share the output. Thanks again for all the help. Thanks and Regards Nishant Aggarwal, PMP Cell No:- +91 99588 94305 On Fri, Aug 21, 2015 at 10:33 AM, Jörn Franke wrote: > Additionally, although it is a PoC you should have a realistic d

Re: Using json_tuple for Nested json Arrays

2015-10-27 Thread Nishant Aggarwal
Hello Sam, You can easily achieve this by using elephant-bird.jars in pig. We are also caturing tweets via flume and filter them using pig and elephant-jars. You can find the related jars over internet. Cheers, Nishant Aggarwal On 28 Oct 2015 00:50, "Sam Joe" wrote: > Hi, > >

Re: Using json_tuple for Nested json Arrays

2015-10-27 Thread Nishant Aggarwal
Hello Sam, Please find attached PIG script for the same. You may find the necessary jars below. http://mvnrepository.com/artifact/com.twitter.elephantbird/elephant-bird-pig Note: Same functionality can be achieved in Hive as well. Thanks and Regards Nishant Aggarwal, PMP Cell No:- +91 99588

Compaction in hive

2016-12-06 Thread Nishant Aggarwal
. Need your help in order to implement Compaction, how to implement, what are the pros and cons. Also, is it mandatory to have bucketing to implement compaction? Request you to please help. Thanks and Regards Nishant Aggarwal, PMP Cell No:- +91 99588 94305 <099588%2094305>

Re: Compaction in hive

2016-12-07 Thread Nishant Aggarwal
on orc/rc format or is it applicable to parquet as well. Thanks again for your comments. Best, Nishant Thanks and Regards Nishant Aggarwal, PMP Cell No:- +91 99588 94305 http://in.linkedin.com/pub/nishant-aggarwal/53/698/11b On Wed, Dec 7, 2016 at 12:32 AM, Alan Gates wrote: > W