leewe created HBASE-16457:
-----------------------------
Summary: Can't start HMaster node because a error or interrupted
while splitting logs
Key: HBASE-16457
URL: https://issues.apache.org/jira/browse/HBASE-16457
Project: HBase
Issue Type: Bug
Environment: CDH4.6
HBase1.0
Reporter: leewe
when i start the hbase cluster, the master node can't start with the error as
follow:
2016-08-19 19:06:53,361 ERROR org.apache.hadoop.hbase.executor.EventHandler:
Caught throwable while processing event M_SERVER_SHUTDOWN
java.io.IOException: Server is stopped
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:193)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-08-19 19:06:53,364 ERROR org.apache.hadoop.hbase.executor.EventHandler:
Caught throwable while processing event M_SERVER_SHUTDOWN
java.io.IOException: Server is stopped
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:193)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-08-19 19:06:53,362 ERROR org.apache.hadoop.hbase.executor.EventHandler:
Caught throwable while processing event M_SERVER_SHUTDOWN
java.io.IOException: failed log splitting for
vm-syscloud-s5,60020,1471599889568, will retry
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.resubmit(ServerShutdownHandler.java:346)
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:219)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: error or interrupted while splitting logs in
[hdfs://nameservice1/hbase/WALs/vm-syscloud-s5,60020,1471599889568-splitting]
Task = installed = 31 done = 0 error = 0
at
org.apache.hadoop.hbase.master.SplitLogManager.splitLogDistributed(SplitLogManager.java:291)
at
org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:391)
at
org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:364)
at
org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:286)
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:212)
... 4 more
2016-08-19 19:06:53,361 ERROR org.apache.hadoop.hbase.executor.EventHandler:
Caught throwable while processing event M_SERVER_SHUTDOWN
java.io.IOException: failed log splitting for
vm-syscloud-s3,60020,1471599886484, will retry
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.resubmit(ServerShutdownHandler.java:346)
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:219)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: error or interrupted while splitting logs in
[hdfs://nameservice1/hbase/WALs/vm-syscloud-s3,60020,1471599886484-splitting]
Task = installed = 3 done = 0 error = 0
at
org.apache.hadoop.hbase.master.SplitLogManager.splitLogDistributed(SplitLogManager.java:291)
at
org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:391)
at
org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:364)
at
org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:286)
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:212)
... 4 more
2016-08-19 19:06:53,361 ERROR org.apache.hadoop.hbase.executor.EventHandler:
Caught throwable while processing event M_SERVER_SHUTDOWN
java.io.IOException: failed log splitting for
vm-syscloud-s2,60020,1471599886831, will retry
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.resubmit(ServerShutdownHandler.java:346)
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:219)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: error or interrupted while splitting logs in
[hdfs://nameservice1/hbase/WALs/vm-syscloud-s2,60020,1471599886831-splitting]
Task = installed = 30 done = 0 error = 0
at
org.apache.hadoop.hbase.master.SplitLogManager.splitLogDistributed(SplitLogManager.java:291)
at
org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:391)
at
org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:364)
at
org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:286)
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:212)
... 4 more
2016-08-19 19:06:53,367 ERROR org.apache.hadoop.hbase.executor.EventHandler:
Caught throwable while processing event M_SERVER_SHUTDOWN
java.io.IOException: Server is stopped
at
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:193)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-08-19 19:06:53,579 INFO
org.apache.hadoop.hbase.master.SplitLogManager$TimeoutMonitor: Chore:
SplitLogManager Timeout Monitor was stopped
i try to delete the log files in the HDFS,but i don't have permission because
the files owner user is hbase. i don't know how to solve this problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
