Andrew Mains created HIVE-7805:
----------------------------------
Summary: Support running multiple scans in hbase-handler
Key: HIVE-7805
URL: https://issues.apache.org/jira/browse/HIVE-7805
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.14.0
Reporter: Andrew Mains
Currently, the HiveHBaseTableInputFormat only supports running a single scan.
This can be less efficient than running multiple disjoint scans in certain
cases, particularly when using a composite row key. For instance, given a row
key schema of:
{code}
struct<bucket int, time timestamp>
{code}
if one wants to push down the predicate:
{code}
bucket IN (1, 10, 100) AND timestamp >= 1408333927 AND timestamp < 1408506670
{code}
it's much more efficient to run a scan for each bucket over the time range
(particularly if there's a large amount of data per day). With a single scan,
the MR job has to process the data for all time for buckets in between 1 and
100.
hive should allow HBaseKeyFactory's to decompose a predicate into one or more
scans in order to take advantage of this fact.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
