Re: Automatic replacement of partitions in hive

2014-03-06 Thread Nitin Pawar
Thanks for clarifying that Bryan On Thu, Mar 6, 2014 at 7:55 PM, Bryan Jeffrey wrote: > Nitin, > > #3 will not work. msck repair table does not remove partitions if the > files associated with the partition do not exist. We have successfully > applied #2 in our application. > > Regards, > > Br

Re: Automatic replacement of partitions in hive

2014-03-06 Thread Bryan Jeffrey
Nitin, #3 will not work. msck repair table does not remove partitions if the files associated with the partition do not exist. We have successfully applied #2 in our application. Regards, Bryan Jeffrey On Thu, Mar 6, 2014 at 5:37 AM, Nitin Pawar wrote: > There is no concept called automati

Re: Automatic replacement of partitions in hive

2014-03-06 Thread Nitin Pawar
There is no concept called automatic. Please wait for expert hive gurus to reply before using any of my suggestions Few options which I can think of are 1) Insert overwrite table with dynamic partitions enabled and restricting the partition column values for the date range you want. Cost of this

Automatic replacement of partitions in hive

2014-03-06 Thread Kasi Subrahmanyam
Hi, I have a table in hive which has data of three months old. I have partitioned the data and I got 90 partitions. Now when I get the new data from next day I want to replace the partition 1week old with the new one automatically. Can this partitioning and replacement be done using swoop at the s