Hello, We enabled log compaction on a few topics, as we want to preserve permanently the latest versions of messages published to specific topics. After enabling compaction, the log cleaner thread dies with the same error for the topics we tried it on. It looks like kafka has starting offset that does not exist in the topic (at least that is how I am reading the error). Any ideas on how we can work around this error?
Thanks Rakesh [2016-04-27 15:52:11,306] INFO [kafka-log-cleaner-thread-0], Starting (kafka.log.LogCleaner) [2016-04-27 15:52:11,322] INFO Cleaner 0: Beginning cleaning of log metamorphosis.lineup-0. (kafka.log.LogCleaner) [2016-04-27 15:52:11,323] INFO Cleaner 0: Building offset map for metamorphosis.lineup-0... (kafka.log.LogCleaner) [2016-04-27 15:52:11,415] INFO Cleaner 0: Building offset map for log metamorphosis.lineup-0 for 1 segments in offset range [1553258, 2138466). (kafka.log.LogCleaner) [2016-04-27 15:52:11,435] ERROR [kafka-log-cleaner-thread-0], Error due to (kafka.log.LogCleaner) java.lang.IllegalArgumentException: requirement failed: Last clean offset is 1553258 but segment base offset is 2125968 for log metamorphosis.lineup-0. at scala.Predef$.require(Predef.scala:233) at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:509) at kafka.log.Cleaner.clean(LogCleaner.scala:307) at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:221) at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:199) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) [2016-04-27 15:52:11,436] INFO [kafka-log-cleaner-thread-0], Stopped (kafka.log.LogCleaner)
