Hi Sanjay,
thanks for the info. indeed, I did raise up the log level earlier to avoid
dumping too much info in app log. I got much the same info and exception after
lowering down at INFO. also, I am trying with CDH 4.1.2, except I am running
mapreduce rather than YARN. The exception observe only in mapreduce (MR1), I
can successfully insert the data into hbase by initiating the mapper tasks
follow the insert statement.
forgot to paste also the log observed in datanodes, hereafter the
java.io.IOException read from datanode log file:
---
at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:243)
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:522)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:373)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.Child.main
Cheers,
Jason
On Mar 28, 2013, at 6:18 AM, Sanjay Subramanian wrote:
> If you can run your hive insert data script with debug option u may get
> some clues
>
> /usr/lib/hive/bin/hive -hiveconf hive.root.logger=INFO,console -e "insert
> into dest select * from some_table_same_structure_as_dest limit 10;"
>
> I created a small demo usecase and this is failing for me as well
> The error I get is
> Unable to retrieve URL for Hadoop Task logs. port out of range:999999
> 13/03/27 14:02:32 ERROR exec.Task: Unable to retrieve URL for Hadoop Task
> logs. port out of range:999999
>
> I am using ClouderaManager 4.1.2 Hadoop, Yarn, HDFS, Oozie, Hive and Hue
>
> sanjay
>
>
> On 3/27/13 1:25 PM, "Jason Shih" <hls...@gmail.com> wrote:
>
>>
>> Hi all,
>>
>>
>> I try inserting data into hive table storing by hbase storage handler but
>> fail with the following exception end of mapreduce stage info.
>>
>> however, we didn't have problem accessing HbaseStorageHandler if force
>> accessing YARN rather than mapreduce. (hive: 0.9.0, hbase: 0.92.1).
>>
>> could expert shed some light how we tweak the settings if would like to
>> initiate the mapper tasks to insert data leveraging HBaseStorageHandler.
>> Thanks.
>>
>>
>> hereafter simple hive script tried before:
>>
>> CREATE TABLE dest(num string, name string)
>> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:name")
>> TBLPROPERTIES ("hbase.table.name" = "dest");
>>
>>
>> and the exception:
>>
>> ...
>> 2013-03-28 03:49:41,107 Stage-0 map = 0%, reduce = 0%, Cumulative CPU
>> 0.67 sec
>> 2013-03-28 03:49:42,125 Stage-0 map = 0%, reduce = 0%, Cumulative CPU
>> 0.67 sec
>> 2013-03-28 03:49:43,143 Stage-0 map = 0%, reduce = 0%, Cumulative CPU
>> 0.67 sec
>> 2013-03-28 03:49:44,153 Stage-0 map = 0%, reduce = 0%
>> 2013-03-28 03:49:47,184 Stage-0 map = 100%, reduce = 100%
>> MapReduce Total cumulative CPU time: 670 msec
>> Ended Job = job_201303251402_0005 with errors
>> Error during job, obtaining debugging information...
>> Examining task ID: task_201303251402_0005_m_000002 (and more) from job
>> job_201303251402_0005
>> Exception in thread "Thread-35" java.lang.NullPointerException
>> at
>> org.apache.hadoop.hive.shims.Hadoop23Shims.getTaskAttemptLogUrl(Hadoop23Sh
>> ims.java:44)
>> at
>> org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.getTaskInfos(Jo
>> bDebugger.java:186)
>> at
>> org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.run(JobDebugger
>> .java:142)
>> at java.lang.Thread.run(Thread.java:662)
>> No encryption was performed by peer.
>> FAILED: Execution Error, return code 2 from
>> org.apache.hadoop.hive.ql.exec.MapRedTask
>> MapReduce Jobs Launched:
>> Job 0: Map: 1 Cumulative CPU: 0.67 sec HDFS Read: 0 HDFS Write: 0 FAIL
>>
>>
>>
>> Cheers,
>> Jason
