[
https://issues.apache.org/jira/browse/HIVE-6117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13858837#comment-13858837
]
Brock Noland commented on HIVE-6117:
------------------------------------
Hey guys,
I think I see the issue here. o.a.h.mapreduce.RecordReader has an initialize
method which is called by the mapper. o.a.h.mapred.RecordReader does not have
an initialize method. Hive uses the mapred API so you can see here:
https://github.com/apache/hive/blob/fb63a28cd5fddb5e5c974cab84cd9c3a4155e40d/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java#L178
that we wrap the mapreduce RR from HBase in a mapred RR. Therefore it's hive's
responsibility to call initialize which we are not doing. Therefore the patch
looks correct and I will commit.
> mapreduce.RecordReader instance needs to be initialized
> -------------------------------------------------------
>
> Key: HIVE-6117
> URL: https://issues.apache.org/jira/browse/HIVE-6117
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Attachments: 6117.00.patch, HIVE-6117.0.patch
>
>
> The HBase storage handler makes use of a mapreduce.RecordReader instance but
> does not initialize it when consumed from local context. This results in a
> NPE for some queries, for instance
> {noformat}
> create table hbase_1(key string, age int) stored by
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties (
> "hbase.columns.mapping" = "info:age");
> insert overwrite table hbase_1 select name, SUM(age) from studenttab10k group
> by name;
> select * from hbase_1;
> {noformat}
> The select statement throws the following exception
> {noformat}
> 13/12/18 01:30:32 ERROR CliDriver: Failed with exception
> java.io.IOException:java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:551)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:489)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1494)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
> at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:446)
> at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:456)
> at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:737)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.nextKeyValue(TableRecordReaderImpl.java:196)
> at
> org.apache.hadoop.hbase.mapreduce.TableRecordReader.nextKeyValue(TableRecordReader.java:138)
> at
> org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat$1.next(HiveHBaseTableInputFormat.java:234)
> at
> org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat$1.next(HiveHBaseTableInputFormat.java:193)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:521)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
