Re: Crashed node has Bitcask merge errors on restart

2011-08-05 Thread Jeff Pollard
Hey David, Thanks again for all your help, much appreciated. We've since restored from our backup and the node appears healthy now. I checked the logs and saw successful bitcask merges and read repairs were happening, so all appears well. I also upped the ulimit -n value based on your observati

Re: Crashed node has Bitcask merge errors on restart

2011-08-05 Thread David Smith
On Fri, Aug 5, 2011 at 6:49 AM, Jeff Pollard wrote: > Update: now the node has crashed, due to the following lines in the > sasl-error.log (see below).  I've also attached the crash dump to this > email. > Real quickly though, just to confirm - If we wanted to restore the node from > a recent back

Re: Crashed node has Bitcask merge errors on restart

2011-08-05 Thread Jeff Pollard
Hey David, Thanks for the reply. I'm in the process of downloading our backup data to the node as we speak. I'll restore the bitcask directory to that data and boot the node, and will let you know how it goes. Thanks again for your help. On Fri, Aug 5, 2011 at 6:01 AM, David Smith wrote: > H

Re: Crashed node has Bitcask merge errors on restart

2011-08-05 Thread David Smith
Hi Jeff, I believe you are encountering BZ 1097 (http://issues.basho.com/1097), where a suddenly truncated bitcask file can cause problems when attempting to merge. The truncation is typically the result of underlying O/S or hardware failure and simply means that the last record in a bitcask file