Hey Anthony, At first glance, it looks like you may have run into an open file handle problem. Try increasing the ulimit to at least 1024. There is more info on the wiki here:
http://wiki.basho.com/Open-Files-Limit.html Give that a shot and let us know if the problem persists. Mark On Tue, Feb 1, 2011 at 4:08 PM, Anthony Molinaro <antho...@alumni.caltech.edu> wrote: > Hi, > > I just set up a 4 node cluster mostly vanilla config with the exception > that I specified 1024 partitions, and I'm using a multi-backend with one > entry for bitcask and the default as bitcask (I plan to deploy a cache > backend at some point). I have one bucket which stores a pretty small > payload (key is 36 bytes, value is 36 bytes). > > Things ran fine under light load (~400 get, ~30 puts according to riak-admin > status, so I think that's per minute). Suddenly several nodes (3/4) all > shutdown within a few minutes of each other. They all seem to have > errors like this > > =ERROR REPORT==== 1-Feb-2011::23:50:35 ===^M > Failed to open lock file > /var/lib/riak/bitcask/1156070631091827503657211635254091060470024765440/bitcask.write.lock: > emfile > > then a state machine termination stacktrace. > > I had run cluster_info for all of them a few minutes before and the only > machine which didn't crash was the one I ran cluster_info on. > > Not sure if that was the cause or not. Any ideas what could cause these > errors? > > I can send more info if it would help, but wanted to get the conversation > started before I head home. > > Thanks, > > -Anthony > > -- > ------------------------------------------------------------------------ > Anthony Molinaro <antho...@alumni.caltech.edu> > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
