Out of curiosity, do you really have on the order of 1,986,622,313 elements (I believe elements=keys) in the cf?
Dan From: shimi [mailto:shim...@gmail.com] Sent: February-09-11 15:06 To: user@cassandra.apache.org Subject: Exceptions on 0.7.0 I have a 4 node test cluster were I test the port to 0.7.0 from 0.6.X On 3 out of the 4 nodes I get exceptions in the log. I am using RP. Changes that I did: 1. changed the replication factor from 3 to 4 2. configured the nodes to use Dynamic Snitch 3. RR of 0.33 I run repair on 2 nodes before I noticed the errors. One of them is having the first error and the other the second. I restart the nodes but I still get the exceptions. The following Exception I get from 2 nodes: WARN [CompactionExecutor:1] 2011-02-09 19:50:51,281 BloomFilter.java (line 84) Cannot provide an optimal Bloom Filter for 1986622313 elements (1/4 buckets per element). ERROR [CompactionExecutor:1] 2011-02-09 19:51:10,190 AbstractCassandraDaemon.java (line 91) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.io.IOError: java.io.EOFException at org.apache.cassandra.io.sstable.SSTableIdentityIterator.next(SSTableIdentity Iterator.java:105) at org.apache.cassandra.io.sstable.SSTableIdentityIterator.next(SSTableIdentity Iterator.java:34) at org.apache.commons.collections.iterators.CollatingIterator.set(CollatingIter ator.java:284) at org.apache.commons.collections.iterators.CollatingIterator.least(CollatingIt erator.java:326) at org.apache.commons.collections.iterators.CollatingIterator.next(CollatingIte rator.java:230) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.jav a:68) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator .java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131 ) at com.google.common.collect.Iterators$7.computeNext(Iterators.java:604) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator .java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131 ) at org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:7 6) at org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50) at org.apache.cassandra.io.LazilyCompactedRow.<init>(LazilyCompactedRow.java:88 ) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterato r.java:136) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.jav a:107) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.jav a:42) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.jav a:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator .java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131 ) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(Filter Iterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterat or.java:94) at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.jav a:323) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja va:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9 08) at java.lang.Thread.run(Thread.java:619) Caused by: java.io.EOFException at java.io.RandomAccessFile.readFully(RandomAccessFile.java:383) at org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:280) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:7 6) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:3 5) at org.apache.cassandra.io.sstable.SSTableIdentityIterator.next(SSTableIdentity Iterator.java:101) ... 29 more On another node I get: ERROR [pool-1-thread-2] 2011-02-09 19:48:32,137 Cassandra.java (line 2876) Internal error processing get_range_ slices java.lang.RuntimeException: error reading 1 of 1970563183 at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleS liceReader.java:82) at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleS liceReader.java:39) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator .java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131 ) at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableS liceIterator.java:108) at org.apache.commons.collections.iterators.CollatingIterator.anyHasNext(Collat ingIterator.java:364) at org.apache.commons.collections.iterators.CollatingIterator.hasNext(Collating Iterator.java:217) at org.apache.cassandra.db.RowIteratorFactory$3.getReduced(RowIteratorFactory.j ava:136) at org.apache.cassandra.db.RowIteratorFactory$3.getReduced(RowIteratorFactory.j ava:106) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.jav a:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator .java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131 ) at org.apache.cassandra.db.RowIterator.hasNext(RowIterator.java:49) at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.ja va:1294) at org.apache.cassandra.service.StorageProxy.getRangeSlice(StorageProxy.java:43 8) at org.apache.cassandra.thrift.CassandraServer.get_range_slices(CassandraServer .java:473) at org.apache.cassandra.thrift.Cassandra$Processor$get_range_slices.process(Cas sandra.java:2868) at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2555) at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(Custom TThreadPoolServer.java:1 67) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja va:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9 08) at java.lang.Thread.run(Thread.java:619) Caused by: java.io.EOFException at java.io.RandomAccessFile.readFully(RandomAccessFile.java:383) at org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:280) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:9 4) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:3 5) at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleS liceReader.java:78) ... 21 more any idea what went wrong? Shimi No virus found in this incoming message. Checked by AVG - www.avg.com Version: 9.0.872 / Virus Database: 271.1.1/3432 - Release Date: 02/09/11 02:34:00
