Also I must add that grepping the logs for a particular file I see this:
INFO [CompactionExecutor:19] 2015-02-24 10:44:35,618 CompactionTask.java (line
120) Compacting
[SSTableReader(path='/data/ranks/positions/ranks-positions-jb-339-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-354-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-408-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-286-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-20-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-127-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-357-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-257-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-316-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-41-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-285-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-338-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-180-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-398-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-249-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-284-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-294-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-248-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-377-Data.db'),
SSTableReader(path='/data/ranks/positions/ranks-positions-jb-395-Data.d ...
also several entries like this in the log after
grep.java.lang.RuntimeException: java.io.FileNotFoundException:
/data/ranks/positions/ranks-positions-jb-41-Data.db (No such file or
directory)Caused by: java.io.FileNotFoundException:
/data/ranks/positions/ranks-positions-jb-41-Data.db (No such file or directory)
I was grepping for jb-41-Data.db ... seems that this file does not exist for
some reason. I must say that when I first added the node I included it's IP in
the seeds list. Then I have decommissioned it, removed it's IP from the seed
list, deleted all data / commit log / saved caches and started it. Since then I
have not manualy deleted any files .
Any ideeas?
On Tuesday, February 24, 2015 11:46 AM, Batranut Bogdan
<batra...@yahoo.com> wrote:
Hello all,
One of my C* throws a big amount of exceptions like this:
ERROR [ReadStage:792] 2015-02-24 10:43:54,183 CassandraDaemon.java (line 199)
Exception in thread Thread[ReadStage:792,5,main]java.lang.RuntimeException:
java.lang.RuntimeException: java.io.FileNotFoundException:
/data/ranks/positions/ranks-positions-jb-174-Data.db (No such file or
directory) at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2008)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)Caused by:
java.lang.RuntimeException: java.io.FileNotFoundException:
/data/ranks/positions/ranks-positions-jb-174-Data.db (No such file or
directory) at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:47)
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile.createReader(CompressedPoolingSegmentedFile.java:48)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:39)
at
org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1239)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.getNextBlock(IndexedSliceReader.java:417)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.fetchMoreData(IndexedSliceReader.java:387)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader.computeNext(IndexedSliceReader.java:145)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader.computeNext(IndexedSliceReader.java:45)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
at
org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.<init>(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:120)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1550)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1379)
at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:333) at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1439)
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2004)
... 3 moreCaused by: java.io.FileNotFoundException:
/data/ranks/positions/ranks-positions-jb-174-Data.db (No such file or
directory) at java.io.RandomAccessFile.open(Native Method) at
java.io.RandomAccessFile.<init>(RandomAccessFile.java:241) at
org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:58)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:76)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:43)
... 29 more
I must say, that this node was started with cleared data / commit log and it
was joining an existing cluster. I see a lot of these exceptions. Not sure what
to do.
