Ray Mattingly created HBASE-28385:
-------------------------------------
Summary: Quota estimates are too optimistic for large scans
Key: HBASE-28385
URL: https://issues.apache.org/jira/browse/HBASE-28385
Project: HBase
Issue Type: Improvement
Reporter: Ray Mattingly
Fix For: 2.6.0
Let's say you're running a table scan with a throttle of 100MB/sec per
RegionServer. Ideally your scans are going to pull down large results, often
containing hundreds or thousands of blocks.
You will estimate each scan as costing a single block of read capacity, and if
your quota is already exhausted then the server will evaluate the backoff
required for your estimated consumption (1 block) to be available. This will
often be ~1ms, causing your retries to basically be immediate.
Obviously it will routinely take much longer than 1ms for 100MB of IO to become
available in the given configuration, so your retries will be destined to fail.
At worst this can cause a saturation of your server's RPC layer, and at best
this causes erroneous exhaustion of the client's retries.
We should find a way to make these estimates a bit smarter for large scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
