Hi,
please check also the permissions in hdfs for hive warehouse-dirs.
best,
Alex
On Tue, Nov 29, 2011 at 12:44 AM, arv...@cloudera.com
<arv...@cloudera.com>wrote:
> Hi Jurgen,
>
> What version of Hive and Sqoop are you using? Also, please look under
> /tmp/${USER}/hive.log file which will have more detailed information on
> what may be going wrong.
>
> Thanks,
> Arvind
>
>
> On Mon, Nov 28, 2011 at 3:17 PM, Jurgen Van Gael <jur...@rangespan.com>wrote:
>
>> Hi,
>> I am running the Cloudera CDH3 Hive distribution in pseudo-distributed
>> mode on my local Mac OS Lion laptop. Hive generally works fine except
>> when I use it together with Sqoop. A command like
>>
>> sqoop import --connect jdbc:mysql://localhost/db --username root
>> --password foobar --table sometable --warehouse-dir
>> /user/hive/warehouse
>>
>> completes succesfully and generates part_files, a _logs directory and
>> a _SUCCESS file in the hive warehouse directory on HDFS. However, when
>> I add the --import-hive part to the Sqoop command, the import still
>> works but Hive seems to get into an infinite loop. Looking at the logs
>> I find entries
>>
>> 2011-11-28 22:54:57,279 WARN org.apache.hadoop.hdfs.StateChange: DIR*
>> FSDirectory.unprotectedRenameTo: failed to rename
>> /user/hive/warehouse/sometable/_SUCCESS to
>> /user/hive/warehouse/sometable/_SUCCESS_copy_2 because source does not
>> exist
>> 2011-11-28 22:54:57,281 WARN org.apache.hadoop.hdfs.StateChange: DIR*
>> FSDirectory.unprotectedRenameTo: failed to rename
>> /user/hive/warehouse/sometable/_SUCCESS to
>> /user/hive/warehouse/sometable/_SUCCESS_copy_3 because source does not
>> exist
>> 2011-11-28 22:54:57,282 WARN org.apache.hadoop.hdfs.StateChange: DIR*
>> FSDirectory.unprotectedRenameTo: failed to rename
>> /user/hive/warehouse/sometable/_SUCCESS to
>> /user/hive/warehouse/sometable/_SUCCESS_copy_4 because source does not
>> exist
>>
>> I started digging into the source code and can trace it back to
>> ql/metadata/Hive.java:checkPaths which tries to find a name for a
>> _SUCCESS file during the actual Hive load but somehow fails because
>> the Sqoop import MR job already created a _SUCCESS file. I already
>> tried disabling MR creation of _SUCCESS files but Hive seems to wait
>> for that file to kick off the Hive import and hence fails.
>>
>> Does anyone have any suggestions on where to search next?
>>
>> Thanks! Jurgen
>>
>
>
--
Alexander Lorenz
http://mapredit.blogspot.com
*P **Think of the environment: please don't print this email unless you
really need to.*
