When I have seen this in the past it has been bad memory on the server. On Thu, Apr 28, 2011 at 11:58 AM, Daniel Doubleday <daniel.double...@gmx.net> wrote: > Hi all > on one of our dev machines we ran into this: > INFO [CompactionExecutor:1] 2011-04-28 15:07:35,174 SSTableWriter.java (line > 108) Last written key : DecoratedKey(12707736894140473154801792860916528374, > 74657374) > INFO [CompactionExecutor:1] 2011-04-28 15:07:35,174 SSTableWriter.java > (line 109) Current key : DecoratedKey(9639328335678452015784116654185025244, > 61c080c080c080c080c080c0803a60) > INFO [CompactionExecutor:1] 2011-04-28 15:07:35,175 SSTableWriter.java > (line 110) Writing into file > /var/lib/cassandra/data/Smeet/Tracker-tmp-f-752-Data.db > ERROR [CompactionExecutor:1] 2011-04-28 15:07:35,179 > AbstractCassandraDaemon.java (line 113) Fatal exception in thread > Thread[CompactionExecutor:1,1,main] > java.io.IOException: Keys must be written in ascending order. > at > org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:111) > at > org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:128) > at > org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:451) > at > org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:124) > at > org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:94) > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:636) > > After looking into this we found that one SSTable was corrupt: > INFO 18:10:23,767 checking sstable > /Users/dd/Temp/cassandra/data/Smeet/Tracker-f-21-Data.db > INFO 18:10:23,773 DecoratedKey(12707736894140473154801792860916528374, > 74657374) > INFO 18:10:23,777 DecoratedKey(9639328335678452015784116654185025244, > 61c080c080c080c080c080c0803a60) > INFO 18:10:23,781 DecoratedKey(14745077953608664589374317792467471221, > 7265706f2d333631343039) > INFO 18:10:23,783 DecoratedKey(20959981347795136420050187639971803838, > 6576656e746c6f672d333631383838) > INFO 18:10:23,785 DecoratedKey(21549594967496343960898734735584772470, > 6576656e746c6f672d333631383837) > INFO 18:10:23,787 DecoratedKey(21604488955263763122920767208831708343, > 6576656e746c6f672d333631383835) > INFO 18:10:23,790 DecoratedKey(50100329769829417461462475380721306089, > 61c080c080c080c080c080c0803a5f) > INFO 18:10:23,792 DecoratedKey(46089976107344533463199626515706634206, > 7265706f2d333631303436) > INFO 18:10:23,794 DecoratedKey(50825113246224326722027014732583502298, > 6c6f674576656e742d333631383431) > INFO 18:10:23,797 DecoratedKey(146509745693074257726394585872920772980, > 61c080c080c080c080c080c0803a59) > INFO 18:10:23,801 DecoratedKey(63662277073629697060427801149510996510, > 6c6f674576656e742d333631383432) > INFO 18:10:23,803 DecoratedKey(79706531464725069255477768601476202854, > 6576656e746c6f672d333631383834) > INFO 18:10:23,808 DecoratedKey(99662740527281958972877086277408797556, > 6c6f674576656e742d333631383430) > INFO 18:10:23,810 DecoratedKey(24216004522287050570821311008920284627, > 61c080c080c080c080c080c0803a5a) > INFO 18:10:23,816 DecoratedKey(130819691046705791898531759480356609485, > 61c080c080c080c080c080c0803a5e) > INFO 18:10:24,121 DecoratedKey(107312182597869061361893705497084751672, > 61c080c080c080c080c080c0803a58) > INFO 18:10:24,123 DecoratedKey(142873550623537979564886292314205248041, > 7265706f2d333631363731) > INFO 18:10:24,124 DecoratedKey(147476468396567670371575440897348610391, > 6576656e746c6f672d333631383836) > I couldn't figure out how SSTableWriter could possible write such a file so > ... Just wanted to report. Maybe it rings a bell somewhere. > It seems really strange that some data corruption would leave the file > readable for the SSTableReader. > History of this server is 0.6.5 -> 0.7.4. But this file was obviously > written by 0.7.4 > > > Cheers, > Daniel > >
-- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
