Chunky, I have used "recover partitions" command on EMR, and that worked fine.
However, take a look at https://issues.apache.org/jira/browse/HIVE-874. Seems like msck command in Apache Hive does the same thing. Try it out and let us know it goes. Mark On Mon, Nov 5, 2012 at 7:56 AM, Edward Capriolo <edlinuxg...@gmail.com>wrote: > Recover partitions should work the same way for different file systems. > > Edward > > On Mon, Nov 5, 2012 at 9:33 AM, Dean Wampler > <dean.wamp...@thinkbiganalytics.com> wrote: > > Writing a script to add the external partitions individually is the only > way > > I know of. > > > > Sent from my rotary phone. > > > > > > On Nov 5, 2012, at 8:19 AM, Chunky Gupta <chunky.gu...@vizury.com> > wrote: > > > > Hi Dean, > > > > Actually I was having Hadoop and Hive cluster on EMR and I have S3 > storage > > containing logs which updates daily and having partition with date(dt). > And > > I was using this recover partition. > > Now I wanted to shift to EC2 and have my own Hadoop and Hive cluster. So, > > what is the alternate of using recover partition in this case, if you > have > > any idea ? > > I found one way of individually partitioning all dates, so I have to > write > > script for that to do so for all dates. Is there any easiest way other > than > > this ? > > > > Thanks, > > Chunky > > > > > > > > On Mon, Nov 5, 2012 at 6:28 PM, Dean Wampler > > <dean.wamp...@thinkbiganalytics.com> wrote: > >> > >> The RECOVER PARTITIONS is an enhancement added by Amazon to their > version > >> of Hive. > >> > >> > >> > http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/emr-hive-additional-features.html > >> > >> <shameless-plus> > >> Chapter 21 of Programming Hive discusses this feature and other > aspects > >> of using Hive in EMR. > >> </shameless-plug> > >> > >> dean > >> > >> > >> On Mon, Nov 5, 2012 at 5:34 AM, Chunky Gupta <chunky.gu...@vizury.com> > >> wrote: > >>> > >>> Hi, > >>> > >>> I am having a cluster setup on EC2 with Hadoop version 0.20.2 and Hive > >>> version 0.8.1 (I configured everything) . I have created a table using > :- > >>> > >>> CREATE EXTERNAL TABLE XXX ( YYY )PARTITIONED BY ( ZZZ )ROW FORMAT > >>> DELIMITED FIELDS TERMINATED BY 'WWW' LOCATION 's3://my-location/data/'; > >>> > >>> Now I am trying to recover partition using :- > >>> > >>> ALTER TABLE XXX RECOVER PARTITIONS; > >>> > >>> but I am getting this error :- "FAILED: Parse Error: line 1:12 cannot > >>> recognize input near 'XXX' 'RECOVER' 'PARTITIONS' in alter table > statement" > >>> > >>> Doing same steps on a cluster setup on EMR with Hadoop version 1.0.3 > and > >>> Hive version 0.8.1 (Configured by EMR), works fine. > >>> > >>> So is this a version issue or am I missing some configuration changes > in > >>> EC2 setup ? > >>> I am not able to find exact solution for this problem on internet. > Please > >>> help me. > >>> > >>> Thanks, > >>> Chunky. > >>> > >>> > >>> > >> > >> > >> > >> -- > >> Dean Wampler, Ph.D. > >> thinkbiganalytics.com > >> +1-312-339-1330 > >> > >> > > >
