I didn't pre-split the table. I checked the master log, there were no splits before the exception, but during the exception, which lasted for dozens of minutes before I restarted the region server, there was a split:
INFO org.apache.hadoop.hbase.master.SplitLogManager: dead splitlog workers ... INFO org.apache.hadoop.hbase.master.SplitLogManager: started splitting 0 logs in [] INFO org.apache.hadoop.hbase.master.SplitLogManager: finished splitting (more than or equal to) 0 bytes in 0 log files in [] in 0ms However, the split seems to have done nothing. So maybe it's not caused by a split, what else may be the cause, was it a bug? P.S. I am using hbase-0.96.1.1+cdh5.0.1+67 with CDH 5.0.1. 2015-04-14 21:40 GMT+08:00 Ted Yu <[email protected]>: > Can you check master log to see if test,1752868451_785774043,1422169937336. > cb3ab092981ff22d51be923bcf7fa81c. split around the time the exception was > thrown ? > > Have you pre-split table test ? > > Which hbase release are you using ? > > Cheers > > On Tue, Apr 14, 2015 at 6:03 AM, Shady Xu <[email protected]> wrote: > > > Our RegionServers kept getting this exception and were therefore unable > to > > response to other requests: > > > > org.apache.hadoop.hbase.regionserver.HRegion: Failed getting lock in > batch > > put, row=... > > org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row > > out of range for row lock on HRegion > > > test,1752868451_785774043,1422169937336.cb3ab092981ff22d51be923bcf7fa81c., > > startKey='1752868451_785774043', getEndKey()='186061631_989857395', > > row='1716676661_1231903641' > > at > > org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:3260) > > at > > > org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:3278) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2213) > > at > > > org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2085) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4050) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3361) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3265) > > at > > > > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26935) > > at > org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175) > > at > > org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1879) > > > > I guess maybe the batch put operation happened during the table split, > Any > > other possible causes and how to prevent it from happening again? > > >
