This sounds like you are being limited by sequentially reading records in a
single thread with multiple queries.
Can you say more about what kind of read your doing and about the structure
of the program initiating the reads?
On Sat, Mar 26, 2011 at 10:01 AM, Hari Sreekumar
wrote:
> So far, I a
On Sat, Mar 26, 2011 at 2:08 AM, Dmitriy Lyubimov wrote:
> or perhaps scratch that. it seems you are saying the problem arises on
> the backend and your reducer code certainly doesn't create over 10
> connections there. so it might be a combination of other tasks running
> at the same address.
>
What is your typical row size? How many column families? How many columns in
each family?
On Mar 26, 2011, at 10:01 AM, Hari Sreekumar wrote:
> Hi guys,
>
> On what factors does HBase read latency primarily depend? What would be the
> approx theoretical limit for read latency in v0.90.1 on a cl
Hi guys,
On what factors does HBase read latency primarily depend? What would be the
approx theoretical limit for read latency in v0.90.1 on a cluster of 7 nodes
(16 core/16 GB RAM on 5 machines and 36 GB on the other two)? I have an
application where I generate around 1000 rows/s to be input into
or perhaps scratch that. it seems you are saying the problem arises on
the backend and your reducer code certainly doesn't create over 10
connections there. so it might be a combination of other tasks running
at the same address.
On Sat, Mar 26, 2011 at 2:04 AM, Dmitriy Lyubimov wrote:
> yes i ha
yes i had a very similar issue although i prefer to think about it in
terms of hbase and by extension zk connection leak in TableInputFormat
rather than adjusting max zk connection to 30 'cause sooner or later
you will run out of it too.
The problem is that HConnectionManager now identifies hbase
