Also, probably best to move this to the HBase mailing list, as this isn't Phoenix-specific.
Thanks, James On Fri, Oct 31, 2014 at 10:05 AM, Jeffrey Zhong <[email protected]> wrote: > > From the following error, it means region 69e9a7efb9ee00b1ecfe50f825e7cc5b > is opening and can¹t serve any request. There are may some issues > preventing the region from being opened. You can check Region Server log > to see why 69e9a7efb9ee00b1ecfe50f825e7cc5b can¹t be opened. > > > On 10/30/14, 10:05 PM, "Li Li" <[email protected]> wrote: > >>I have tried to set replication factor to 1 by hadoop fs -setrep -R 1 >>/ but no luck >> >>On Fri, Oct 31, 2014 at 1:03 PM, Li Li <[email protected]> wrote: >>> my hbase(phoenix) crashes when the disk is full. I killed the hbase >>> and hadoop related processes. But After that I can't start hbase >>> anymore. >>> I am using ubuntu 12.04 and hadoop-1.2.1 and hbase 0.98.5 with phoenix >>>4.1.0 >>> The region server print error like: >>> >>> 2014-10-31 12:33:49,835 INFO >>> [mobvoi-knowledge-graph-0,60020,1414729587199-recovery-writer--pool4-t2] >>> client.AsyncProcess: #9, waiting for some tasks to finish. Expected >>> max=0, tasksSent=31, tasksDone=30, currentTasksDone=30, retries=30 >>> hasError=false, tableName=BAIDUMUSIC.BAIDUMUSIC_IDX >>> >>> 2014-10-31 12:33:49,842 INFO [htable-pool9-t1] client.AsyncProcess: >>> #9, table=BAIDUMUSIC.BAIDUMUSIC_IDX, attempt=31/350 failed 1 ops, last >>> exception: org.apache.hadoop.hbase.exceptions.RegionOpeningException: >>> org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region >>> >>>BAIDUMUSIC.BAIDUMUSIC_IDX,\x0D\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0 >>>0\x00\x00\x00\x00\x00\x00,1414488087936.69e9a7efb9ee00b1ecfe50f825e7cc5b. >>> is opening on mobvoi-knowledge-graph-0,60020,1414729587199 >>> >>> at >>>org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName >>>(HRegionServer.java:2692) >>> >>> at >>>org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServe >>>r.java:4139) >>> >>> at >>>org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.ja >>>va:3363) >>> >>> at >>>org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.c >>>allBlockingMethod(ClientProtos.java:29593) >>> >>> at >>>org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2026) >>> >>> at >>>org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98) >>> >>> at >>>org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114 >>>) >>> >>> at >>>org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) >>> >>> at java.lang.Thread.run(Thread.java:745) >>> >>> on mobvoi-knowledge-graph-0,60020,1414484323072, tracking started Fri >>> Oct 31 12:26:39 CST 2014, retrying after 20119 ms, replay 1 ops. >>> >>> 2014-10-31 12:33:55,747 INFO >>> [mobvoi-knowledge-graph-0,60020,1414729587199-recovery-writer--pool4-t3] >>> client.AsyncProcess: #25, waiting for some tasks to finish. Expected >>> max=0, tasksSent=31, tasksDone=30, currentTasksDone=30, retries=30 >>> hasError=false, tableName=BAIDUMUSIC.BAIDUMUSIC_IDX >>> >>> 2014-10-31 12:33:55,755 INFO [htable-pool24-t1] client.AsyncProcess: >>> #25, table=BAIDUMUSIC.BAIDUMUSIC_IDX, attempt=31/350 failed 2 ops, >>> last exception: >>> org.apache.hadoop.hbase.exceptions.RegionOpeningException: >>> org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region >>> >>>BAIDUMUSIC.BAIDUMUSIC_IDX,\x0C\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0 >>>0\x00\x00\x00\x00\x00\x00,1414488087936.378bef2d741a9c00761160670329fc7c. >>> is opening on mobvoi-knowledge-graph-0,60020,1414729587199 >>> >>> at >>>org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName >>>(HRegionServer.java:2692) >>> >>> at >>>org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServe >>>r.java:4139) >>> >>> at >>>org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.ja >>>va:3363) >>> >>> at >>>org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.c >>>allBlockingMethod(ClientProtos.java:29593) >>> >>> at >>>org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2026) >>> >>> at >>>org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98) >>> >>> at >>>org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114 >>>) >>> >>> at >>>org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) >>> >>> at java.lang.Thread.run(Thread.java:745) >>> >>> on mobvoi-knowledge-graph-0,60020,1414484323072, tracking started Fri >>> Oct 31 12:26:45 CST 2014, retrying after 20001 ms, replay 2 ops. >>> >>> I also find warns like >>> wal.FSHLog: HDFS pipeline error detected. Found 1 replicas but >>> expecting no less than 3 replicas. Requesting close of hlog. >>> But my hbase cluster is pesudo cluster that all processes are running >>> in a single machine and the hdfs (I have set dfs.replication=1 in >>> hdfs-site.xml) > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
