Re: java.lang.OutOfMemoryError: Map failed

[jeff]

Daniel:

 

Thanks. That thread helped me solve my problem.

 

I was able to run a 700k MySQL record import without a single memory error.

 

I changed the following sections in storage-conf.xml to fix the OutofMemory errors:

 

 <DiskAccessMode>standard</DiskAccessMode>

<CommitLogSync>batch</CommitLogSync>

 

 <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS>

-----Daniel <dani...@outbrain.com> wrote: -----

To: "user@cassandra.apache.org" <user@cassandra.apache.org>
From: Daniel <dani...@outbrain.com>
Date: 06/21/2010 02:30PM
Subject: Re: java.lang.OutOfMemoryError: Map failed

I am no expert in Cassandra, but it looks like you might get your answer from reading this thread:
http://www.mail-archive.com/user@cassandra.apache.org/msg03702.html

Daniel.

On 06/21/2010 06:35 PM, j...@javajet.com wrote:

I am using Lucandra to write Lucene documents to my cassandra server. I am processing a MySQL table of about 700k records, 10k at a time. All goes well until I reach about 220k mark. Figure it has something to do with my lack of correct memory configuration for JVM, keyspace or Cassandra.

The Map failed is thrown for many *.db files and Cassandra fails to start with java.lang.OutOfMemoryError: unable to create new native thread message.

ERROR 11:31:49,137 Corrupt file /var/lib/cassandra/data/Lucandra/TermInfo-256-Data.db; skipped
java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:819)
        at org.apache.cassandra.io.SSTableReader.mmap(SSTableReader.java:208)
        at org.apache.cassandra.io.SSTableReader.<init>(SSTableReader.java:154)
        at org.apache.cassandra.io.SSTableReader.<init>(SSTableReader.java:218)
        at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:123)
        at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:178)
        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
        at org.apache.cassandra.db.Table.<init>(Table.java:338)
        at org.apache.cassandra.db.Table.open(Table.java:199)
        at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:816)
        ... 11 more
 INFO 11:31:49,140 Replaying /var/lib/cassandra/commitlog/CommitLog-1277133857874.log
ERROR 11:31:49,146 Exception encountered during startup.
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:614)
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:943)
        at java.util.concurrent.ThreadPoolExecutor.prestartAllCoreThreads(ThreadPoolExecutor.java:1549)
        at org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor.<init>(JMXEnabledThreadPoolExecutor.java:51)
        at org.apache.cassandra.concurrent.JMXConfigurableThreadPoolExecutor.<init>(JMXConfigurableThreadPoolExecutor.java:34)
        at org.apache.cassandra.concurrent.StageManager.multiThreadedConfigurableStage(StageManager.java:81)
        at org.apache.cassandra.concurrent.StageManager.<clinit>(StageManager.java:53)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:259)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:173)
        at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:95)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
Exception encountered during startup.


Jeffrey Griffin, Developer
JavaJet Consulting  LLC
 o: 1.717.533.5557
m: 1.717.419.2869
skype: javajet.coder