On Dec 10, 2010, at 19:37, Peter Schuller wrote:
> To cargo cult it: Are you running a modern JVM? (Not e.g. openjdk b17
> in lenny or some such.) If it is a JVM issue, ensuring you're using a
> reasonably recent JVM is probably much easier than to start tracking
> it down...
I had OOM problems with OpenJDK, switched to Sun/Oracle's recent 1.6.0_23
and...still have the same problem :-\ Stack trace always looks the same:
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:329)
at
org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:261)
at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:76)
at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
at
org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129)
at
org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:120)
at
org.apache.cassandra.db.RowMutationSerializer.defreezeTheMaps(RowMutation.java:383)
at
org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:393)
at
org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:351)
at
org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:52)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:63)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
I'm writing from 1 client with 50 threads to a cluster of 4 machines (with
hector). With QUORUM and ONE 2 machines quite reliably will soon die with OOM.
What may cause this? Won't cassandra block/reject when memtable is full and
being flushed to disk but grow and if flushing to disk isn't fast enough will
run out of memory?
