The index files work in the following way : Its a mapping from logical offsets to a particular file location within the log file segment.
If you see the comments under OffsetIndex.scala code : The file format is a series of entries. The physical format is a 4 byte "relative" offset and a 4 byte file location for the message with that offset. The offset stored is relative to the base offset of the index file. So, for example, if the base offset was 50, then the offset 55 would be stored as 5. Using relative offsets in this way let's us use only 4 bytes for the offset. In you case the index file is non empty and its expecting that the base offset should be greater than the last offset and that why it throws the error. I suppose you can get rid of those index files and upon restart those index will be rebuilt. Thanks, Mayuresh On Fri, Mar 13, 2015 at 3:38 PM, Zakee <kzak...@netzero.net> wrote: > I did a shutdown of the cluster and then try to restart and see the below > error on one of the 5 brokers, I can’t restart this instance and not sure > how to fix this. > > [2015-03-13 15:27:31,793] ERROR There was an error in one of the threads > during logs loading: java.lang.IllegalArgumentException: requirement > failed: Corrupt index found, index file > (/data/vol6/kafka/kafka82/Topic22-6/00000000000006256447.index) has > non-zero size but the last offset is 6256447 and the base offset is 6256447 > (kafka.log.LogManager) > [2015-03-13 15:27:31,795] FATAL [Kafka Server 5], Fatal error during > KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) > java.lang.IllegalArgumentException: requirement failed: Corrupt index > found, index file > (/data/vol6/kafka/kafka82/Topic22-6/00000000000006256447.index) has > non-zero size but the last offset is 6256447 and the base offset is 6256447 > at scala.Predef$.require(Predef.scala:233) > at kafka.log.OffsetIndex.sanityCheck(OffsetIndex.scala:352) > at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:184) > at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:183) > at scala.collection.Iterator$class.foreach(Iterator.scala:727) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) > at > scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > at scala.collection.AbstractIterable.foreach(Iterable.scala:54) > at kafka.log.Log.loadSegments(Log.scala:183) > at kafka.log.Log.<init>(Log.scala:67) > at > kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$7$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:142) > at kafka.utils.Utils$$anon$1.run(Utils.scala:54) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > [2015-03-13 15:27:31,831] INFO [Kafka Server 5], shutting down > (kafka.server.KafkaServer) > > > Thanks > Zakee > > > > > ____________________________________________________________ > Fast, Secure, NetZero 4G Mobile Broadband. Try it. > http://www.netzero.net/?refcd=NZINTISP0512T4GOUT2 > -- -Regards, Mayuresh R. Gharat (862) 250-7125
