How balanced are the table regions in your cluster ? Cheers
On Thu, Aug 27, 2015 at 6:15 AM, donmai <[email protected]> wrote: > I figured out the issue - the reason wasn't actually region movement taking > a while, the balancer is actually the thing taking forever: > > 2015-08-27 12:50:13,582 DEBUG [hostname,60000,1440642755872-BalancerChore] > balancer.StochasticLoadBalancer: Could not find a better load balance > plan. Tried 0 different configurations in 2211294ms, and did not find > anything with a computed cost less than 54.18640355329625 > > After waiting for half an hour to an hour, only one region is ever moved by > the balancer and this process is repeated. I'm using default settings with > regard to slop / overall balancing...any idea why it's taking so long? > Thanks! > > On Tue, Aug 25, 2015 at 4:46 PM, anil gupta <[email protected]> wrote: > > > AFAIK, region movement does not moves the data of region on the > > (distributed)FileSystem. It should only, update metadata of HBase. > > Did you check diskio stats during region movement? > > > > On Tue, Aug 25, 2015 at 10:40 AM, Ted Yu <[email protected]> wrote: > > > > > Please see http://hbase.apache.org/book.html#regions.arch.assignment > > > > > > On Tue, Aug 25, 2015 at 10:37 AM, donmai <[email protected]> wrote: > > > > > > > NFS > > > > 0.98.10 > > > > Will get to you as soon as I am able, on travel > > > > > > > > Is my general understanding correct, though, that there shouldn't be > > any > > > > data movement from a region reassignment? > > > > > > > > On Tue, Aug 25, 2015 at 12:40 PM, Ted Yu <[email protected]> > wrote: > > > > > > > > > Can you give a bit more information: > > > > > > > > > > which filesystem you use > > > > > which hbase release you use > > > > > master log snippet for the long region assignment > > > > > > > > > > Thanks > > > > > > > > > > On Tue, Aug 25, 2015 at 9:30 AM, donmai <[email protected]> wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I'm curious about how exactly region movement works with regard > to > > > data > > > > > > transfer. To my understanding from the docs given an HDFS-backed > > > > > cluster, a > > > > > > region movement / transition involves changing things in meta > only, > > > all > > > > > > data movement for locality is handled by HDFS. In the case where > > > > rootdir > > > > > is > > > > > > a shared file system, there shouldn't be any data movement with a > > > > region > > > > > > reassignment, correct? I'm running into performance issues where > > > region > > > > > > assignment takes a very long time and I'm trying to figure out > why. > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Thanks & Regards, > > Anil Gupta > > >
