Hi Mark,
I didn't get any error.
I ran this on hive console:-
"msck table Table_Name;"
It says Ok and showed the execution time as 1.050 sec.
But when I checked partitions for table using
"show partitions Table_Name;"
It didn't show me any partitions.
Thanks,
Chunky.
On Tue, Nov 6, 2012 at 10:38 PM, Mark Grover <grover.markgro...@gmail.com>wrote:
> Glad to hear, Chunky.
>
> Out of curiosity, what errors did you get when using msck?
>
>
> On Tue, Nov 6, 2012 at 5:14 AM, Chunky Gupta <chunky.gu...@vizury.com>wrote:
>
>> Hi Mark,
>> I tried msck, but it is not working for me. I have written a python
>> script to partition the data individually.
>>
>> Thank you Edward, Mark and Dean.
>> Chunky.
>>
>>
>> On Mon, Nov 5, 2012 at 11:08 PM, Mark Grover <grover.markgro...@gmail.com
>> > wrote:
>>
>>> Chunky,
>>> I have used "recover partitions" command on EMR, and that worked fine.
>>>
>>> However, take a look at https://issues.apache.org/jira/browse/HIVE-874.
>>> Seems
>>> like msck command in Apache Hive does the same thing. Try it out and let us
>>> know it goes.
>>>
>>> Mark
>>>
>>> On Mon, Nov 5, 2012 at 7:56 AM, Edward Capriolo
>>> <edlinuxg...@gmail.com>wrote:
>>>
>>>> Recover partitions should work the same way for different file systems.
>>>>
>>>> Edward
>>>>
>>>> On Mon, Nov 5, 2012 at 9:33 AM, Dean Wampler
>>>> <dean.wamp...@thinkbiganalytics.com> wrote:
>>>> > Writing a script to add the external partitions individually is the
>>>> only way
>>>> > I know of.
>>>> >
>>>> > Sent from my rotary phone.
>>>> >
>>>> >
>>>> > On Nov 5, 2012, at 8:19 AM, Chunky Gupta <chunky.gu...@vizury.com>
>>>> wrote:
>>>> >
>>>> > Hi Dean,
>>>> >
>>>> > Actually I was having Hadoop and Hive cluster on EMR and I have S3
>>>> storage
>>>> > containing logs which updates daily and having partition with
>>>> date(dt). And
>>>> > I was using this recover partition.
>>>> > Now I wanted to shift to EC2 and have my own Hadoop and Hive cluster.
>>>> So,
>>>> > what is the alternate of using recover partition in this case, if you
>>>> have
>>>> > any idea ?
>>>> > I found one way of individually partitioning all dates, so I have to
>>>> write
>>>> > script for that to do so for all dates. Is there any easiest way
>>>> other than
>>>> > this ?
>>>> >
>>>> > Thanks,
>>>> > Chunky
>>>> >
>>>> >
>>>> >
>>>> > On Mon, Nov 5, 2012 at 6:28 PM, Dean Wampler
>>>> > <dean.wamp...@thinkbiganalytics.com> wrote:
>>>> >>
>>>> >> The RECOVER PARTITIONS is an enhancement added by Amazon to their
>>>> version
>>>> >> of Hive.
>>>> >>
>>>> >>
>>>> >>
>>>> http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/emr-hive-additional-features.html
>>>> >>
>>>> >> <shameless-plus>
>>>> >> Chapter 21 of Programming Hive discusses this feature and other
>>>> aspects
>>>> >> of using Hive in EMR.
>>>> >> </shameless-plug>
>>>> >>
>>>> >> dean
>>>> >>
>>>> >>
>>>> >> On Mon, Nov 5, 2012 at 5:34 AM, Chunky Gupta <
>>>> chunky.gu...@vizury.com>
>>>> >> wrote:
>>>> >>>
>>>> >>> Hi,
>>>> >>>
>>>> >>> I am having a cluster setup on EC2 with Hadoop version 0.20.2 and
>>>> Hive
>>>> >>> version 0.8.1 (I configured everything) . I have created a table
>>>> using :-
>>>> >>>
>>>> >>> CREATE EXTERNAL TABLE XXX ( YYY )PARTITIONED BY ( ZZZ )ROW FORMAT
>>>> >>> DELIMITED FIELDS TERMINATED BY 'WWW' LOCATION
>>>> 's3://my-location/data/';
>>>> >>>
>>>> >>> Now I am trying to recover partition using :-
>>>> >>>
>>>> >>> ALTER TABLE XXX RECOVER PARTITIONS;
>>>> >>>
>>>> >>> but I am getting this error :- "FAILED: Parse Error: line 1:12
>>>> cannot
>>>> >>> recognize input near 'XXX' 'RECOVER' 'PARTITIONS' in alter table
>>>> statement"
>>>> >>>
>>>> >>> Doing same steps on a cluster setup on EMR with Hadoop version
>>>> 1.0.3 and
>>>> >>> Hive version 0.8.1 (Configured by EMR), works fine.
>>>> >>>
>>>> >>> So is this a version issue or am I missing some configuration
>>>> changes in
>>>> >>> EC2 setup ?
>>>> >>> I am not able to find exact solution for this problem on internet.
>>>> Please
>>>> >>> help me.
>>>> >>>
>>>> >>> Thanks,
>>>> >>> Chunky.
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Dean Wampler, Ph.D.
>>>> >> thinkbiganalytics.com
>>>> >> +1-312-339-1330
>>>> >>
>>>> >>
>>>> >
>>>>
>>>
>>>
>>
>
