try increasing vm.max_map_count per http://blog.timstoop.nl/2011/04/20/cassandra-java-io-ioerror-java-io-ioexception-map-failed/
Feng Qu >________________________________ > From: Raj N <raj.cassan...@gmail.com> >To: user@cassandra.apache.org >Sent: Tuesday, September 18, 2012 6:37 PM >Subject: Cassandra 0.8.4 node keeps crashing with OOM errors > > >One of our nodes keeps crashing continuously with out of memory errors. I see >the following error in the logs - > > > INFO 21:03:54,007 Creating new commitlog segment >/local3/logs/cassandra/commitlog/CommitLog-1348016634007.log >Java HotSpot(TM) 64-Bit Server VM warning: Attempt to allocate stack guard >pages failed. >Java HotSpot(TM) 64-Bit Server VM warning: Attempt to allocate stack guard >pages failed. > INFO 21:03:54,017 Replaying >/local3/logs/cassandra/commitlog/CommitLog-1347950471775.log, >/local3/logs/cassandra/commitlog/CommitLog-1347952280335.log, >/local3/logs/cassandra/commitlog/CommitLog- >1347954861373.log, >/local3/logs/cassandra/commitlog/CommitLog-1347957791957.log, >/local3/logs/cassandra/commitlog/CommitLog-1347959168686.log, >/local3/logs/cassandra/commitlog/CommitLog-1347961014 >948.log, /local3/logs/cassandra/commitlog/CommitLog-1347968405068.log, >/local3/logs/cassandra/commitlog/CommitLog-1347972420459.log, >/local3/logs/cassandra/commitlog/CommitLog-1347975084979.log, / >local3/logs/cassandra/commitlog/CommitLog-1347975538081.log, >/local3/logs/cassandra/commitlog/CommitLog-1347976033450.log, >/local3/logs/cassandra/commitlog/CommitLog-1347976685447.log, /local3/log >s/cassandra/commitlog/CommitLog-1347977204225.log, >/local3/logs/cassandra/commitlog/CommitLog-1347977904344.log, >/local3/logs/cassandra/commitlog/CommitLog-1347978791835.log, >/local3/logs/cassandr >a/commitlog/CommitLog-1347979595214.log, >/local3/logs/cassandra/commitlog/CommitLog-1347980280043.log, >/local3/logs/cassandra/commitlog/CommitLog-1347980822272.log, >/local3/logs/cassandra/commitlo >g/CommitLog-1347981376135.log, >/local3/logs/cassandra/commitlog/CommitLog-1347982023403.log, >/local3/logs/cassandra/commitlog/CommitLog-1347982906942.log, >/local3/logs/cassandra/commitlog/CommitLo >g-1347983999163.log, >/local3/logs/cassandra/commitlog/CommitLog-1347985475186.log, >/local3/logs/cassandra/commitlog/CommitLog-1347987026500.log, >/local3/logs/cassandra/commitlog/CommitLog-13479885 >20038.log, /local3/logs/cassandra/commitlog/CommitLog-1347990191955.log, >/local3/logs/cassandra/commitlog/CommitLog-1347991899744.log, >/local3/logs/cassandra/commitlog/CommitLog-1347993488753.log, > /local3/logs/cassandra/commitlog/CommitLog-1347994801952.log, >/local3/logs/cassandra/commitlog/CommitLog-1347995807921.log, >/local3/logs/cassandra/commitlog/CommitLog-1347996959596.log, /local3/l >ogs/cassandra/commitlog/CommitLog-1347997961248.log, >/local3/logs/cassandra/commitlog/CommitLog-1348005812088.log, >/local3/logs/cassandra/commitlog/CommitLog-1348015418757.log >ERROR 21:03:54,030 Exception encountered during startup. >java.lang.OutOfMemoryError > at java.io.RandomAccessFile.readBytes(Native Method) > at java.io.RandomAccessFile.read(RandomAccessFile.java:322) > at >org.apache.cassandra.io.util.BufferedRandomAccessFile.reBuffer(BufferedRandomAccessFile.java:254) > at >org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:124) > at >org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:196) > at >org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:158) > at >org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:195) > at >org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:336) > at >org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:91) >Exception encountered during startup. >java.lang.OutOfMemoryError > at java.io.RandomAccessFile.readBytes(Native Method) > at java.io.RandomAccessFile.read(RandomAccessFile.java:322) > at >org.apache.cassandra.io.util.BufferedRandomAccessFile.reBuffer(BufferedRandomAccessFile.java:254) > at >org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:124) > at >org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:196) > at >org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:158) > at >org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:195) > at >org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:336) > at >org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:91) >Java HotSpot(TM) 64-Bit Server VM warning: Attempt to allocate stack guard >pages failed. > > > > >Has anybody seen this before? > > >-Rajesh > >
