Hi, I have got a collection of HBase clusters. Each cluster is running separate instances of Zookeeper, Hadoop & HBase. The clusters are either single node or three node setups.
I am getting constant stability problems with the HBase Regionserver, it dies randomly everyday or every other day. It normally dies shortly after printing the following: ERROR [Thread-125066] hdfs.DFSClient (DFSClient.java:closeAllFilesBeingWritten(911)) - Failed to close inode 32621 org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/WALs/extras1.ci.local,60020,1417171049368/extras1.ci.local%2C60020%2C1417171049368.1417295579753 could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and no node(s) are excluded in this operation. Does anyone have any pointers on where I can look to debug this issue? thanks Rob Registered name: In Practice Systems Ltd. Registered address: The Bread Factory, 1a Broughton Street, London, SW8 3QJ Registered Number: 1788577 Registered in England Visit our Internet Web site at www.inps.co.uk The information in this internet email is confidential and is intended solely for the addressee. Access, copying or re-use of information in it by anyone else is not authorised. Any views or opinions presented are solely those of the author and do not necessarily represent those of INPS or any of its affiliates. If you are not the intended recipient please contact [email protected]
