Hi list, We run Cassandra 1.2 on three-node cluster. Each node has 16GB RAM, single 200GB HDD with Ubuntu Server 12.04.
There is an issue with one table that contains about 3000 rows, here its describe-table: CREATE TABLE outputs ( appid text, staged boolean, field ascii, data blob, PRIMARY KEY (appid, staged, field) ) WITH bloom_filter_fp_chance=0.010000 AND caching='KEYS_ONLY' AND comment='' AND dclocal_read_repair_chance=0.000000 AND gc_grace_seconds=864000 AND read_repair_chance=0.100000 AND replicate_on_write='true' AND populate_io_cache_on_flush='false' AND compaction={'class': 'SizeTieredCompactionStrategy'} AND compression={'sstable_compression': 'SnappyCompressor'}; Column DATA contains blobs of size about 1..50MB, average size should be something of 5MB. Sometimes this table expiriences huge write loads for few hours, at such times I see suspicious things in logs: ERROR [CompactionExecutor:357] 2013-07-24 12:32:10,293 CassandraDaemon.java (line 192) Exception in thread Thread[CompactionExecutor:357,1,main] java.lang.AssertionError: incorrect row data size 172489604 written to /var/lib/cassandra/data/woodpecker/outputs/woodpecker-outputs-tmp-ic-813-Data.db; correct is 172489704 at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:162) at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:162) at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58) at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60) at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:211) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) What shall I do about this? Thanks in advance. Pavel.