Re: Broker Restart failed w/ Corrupt index found

2015-03-14 Thread Zakee
I will try to reproduce by repeating the steps I remember, the next time when I restart the cluster. Thanks Zakee > On Mar 13, 2015, at 4:34 PM, Jiangjie Qin wrote: > > Can you reproduce this problem? Although the the fix is strait forward we > would like to understand why this happened. >

Re: Broker Restart failed w/ Corrupt index found

2015-03-13 Thread Zakee
Thanks, Mayuresh. I did the same and it fixed the issue. Thanks Zakee > On Mar 13, 2015, at 3:56 PM, Mayuresh Gharat > wrote: > > The index files work in the following way : > Its a mapping from logical offsets to a particular file location within the > log file segment. > > If you see the

Re: Broker Restart failed w/ Corrupt index found

2015-03-13 Thread Jiangjie Qin
Can you reproduce this problem? Although the the fix is strait forward we would like to understand why this happened. On 3/13/15, 3:56 PM, "Zakee" wrote: >Just found there is a known issue to be resolved in future kafka version: > https://issues.apache.org/jira/browse/KAFKA-1554 > >The workaroun

Re: Broker Restart failed w/ Corrupt index found

2015-03-13 Thread Zakee
Just found there is a known issue to be resolved in future kafka version: https://issues.apache.org/jira/browse/KAFKA-1554 The workaround mentioned here helped. > The workaround is to delete all index files of size 10MB (the size of the > pre-allocated files), and restart. Index files will be

Re: Broker Restart failed w/ Corrupt index found

2015-03-13 Thread Mayuresh Gharat
The index files work in the following way : Its a mapping from logical offsets to a particular file location within the log file segment. If you see the comments under OffsetIndex.scala code : The file format is a series of entries. The physical format is a 4 byte "relative" offset and a 4 byte f

Broker Restart failed w/ Corrupt index found

2015-03-13 Thread Zakee
I did a shutdown of the cluster and then try to restart and see the below error on one of the 5 brokers, I can’t restart this instance and not sure how to fix this. [2015-03-13 15:27:31,793] ERROR There was an error in one of the threads during logs loading: java.lang.IllegalArgumentException: