Hi all
on one of our dev machines we ran into this:
INFO [CompactionExecutor:1] 2011-04-28 15:07:35,174 SSTableWriter.java (line
108) Last written key : DecoratedKey(12707736894140473154801792860916528374,
74657374)
INFO [CompactionExecutor:1] 2011-04-28 15:07:35,174 SSTableWriter.java (line
109) Current key : DecoratedKey(9639328335678452015784116654185025244,
61c080c080c080c080c080c0803a60)
INFO [CompactionExecutor:1] 2011-04-28 15:07:35,175 SSTableWriter.java (line
110) Writing into file /var/lib/cassandra/data/Smeet/Tracker-tmp-f-752-Data.db
ERROR [CompactionExecutor:1] 2011-04-28 15:07:35,179
AbstractCassandraDaemon.java (line 113) Fatal exception in thread
Thread[CompactionExecutor:1,1,main]
java.io.IOException: Keys must be written in ascending order.
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:111)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:128)
at
org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:451)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:124)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:94)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
After looking into this we found that one SSTable was corrupt:
INFO 18:10:23,767 checking sstable
/Users/dd/Temp/cassandra/data/Smeet/Tracker-f-21-Data.db
INFO 18:10:23,773 DecoratedKey(12707736894140473154801792860916528374,
74657374)
INFO 18:10:23,777 DecoratedKey(9639328335678452015784116654185025244,
61c080c080c080c080c080c0803a60)
INFO 18:10:23,781 DecoratedKey(14745077953608664589374317792467471221,
7265706f2d333631343039)
INFO 18:10:23,783 DecoratedKey(20959981347795136420050187639971803838,
6576656e746c6f672d333631383838)
INFO 18:10:23,785 DecoratedKey(21549594967496343960898734735584772470,
6576656e746c6f672d333631383837)
INFO 18:10:23,787 DecoratedKey(21604488955263763122920767208831708343,
6576656e746c6f672d333631383835)
INFO 18:10:23,790 DecoratedKey(50100329769829417461462475380721306089,
61c080c080c080c080c080c0803a5f)
INFO 18:10:23,792 DecoratedKey(46089976107344533463199626515706634206,
7265706f2d333631303436)
INFO 18:10:23,794 DecoratedKey(50825113246224326722027014732583502298,
6c6f674576656e742d333631383431)
INFO 18:10:23,797 DecoratedKey(146509745693074257726394585872920772980,
61c080c080c080c080c080c0803a59)
INFO 18:10:23,801 DecoratedKey(63662277073629697060427801149510996510,
6c6f674576656e742d333631383432)
INFO 18:10:23,803 DecoratedKey(79706531464725069255477768601476202854,
6576656e746c6f672d333631383834)
INFO 18:10:23,808 DecoratedKey(99662740527281958972877086277408797556,
6c6f674576656e742d333631383430)
INFO 18:10:23,810 DecoratedKey(24216004522287050570821311008920284627,
61c080c080c080c080c080c0803a5a)
INFO 18:10:23,816 DecoratedKey(130819691046705791898531759480356609485,
61c080c080c080c080c080c0803a5e)
INFO 18:10:24,121 DecoratedKey(107312182597869061361893705497084751672,
61c080c080c080c080c080c0803a58)
INFO 18:10:24,123 DecoratedKey(142873550623537979564886292314205248041,
7265706f2d333631363731)
INFO 18:10:24,124 DecoratedKey(147476468396567670371575440897348610391,
6576656e746c6f672d333631383836)
I couldn't figure out how SSTableWriter could possible write such a file so ...
Just wanted to report. Maybe it rings a bell somewhere.
It seems really strange that some data corruption would leave the file readable
for the SSTableReader.
History of this server is 0.6.5 -> 0.7.4. But this file was obviously written
by 0.7.4
Cheers,
Daniel
