Thanks for the reply and explanation, Jun ! So yes, I was running the DumpLogSegment tool on an active timeindex segment. But among two clusters i tested with, I did not get the same error on the destination clutser (which is mirroring data from source cluster) even when I ran the DumpLogSegment tool on an active timeindex file. So thats what got me curious.
10485756 Jan 23 15:56 00000000000001690648.timeindex [this is the active timeindex] $ sudo kafka-run-class kafka.tools.DumpLogSegments --files /mnt/data/kafka-logs/upgrade_test-0/00000000000001690648.timeindex Dumping /mnt/data/kafka-logs/upgrade_test-0/00000000000001690648.timeindex timestamp: 0 offset: 1690648 timestamp: 0 offset: 1690648 timestamp: 0 offset: 1690648 timestamp: 0 offset: 1690648 timestamp: 0 offset: 1690648 Thanks, Meghana On Fri, Jan 20, 2017 at 5:07 PM, Jun Rao <j...@confluent.io> wrote: > Meghana, > > You are probably seeing this when running the DumpLogSegment tool on the > active (last) log segment. DumpLogSegment tool is supposed to only be used > on the index of the immutable segments (i.e., when they are rolled). We > preallocate the index on the active segment with 0 values. So, > DumpLogSegment doesn't know the actual valid entries in the index and will > report those 0 values as errors. If you wait for the log segment to roll, > you should see the timeindex resized to 0 length. > > Thanks, > > Jun > > On Fri, Jan 20, 2017 at 8:20 AM, Meghana Narasimhan < > mnarasim...@bandwidth.com> wrote: > > > Hi, > > I'm testing upgrading our cluster from 0.9.0.1 to 0.10.1.0 on 2 clusters > A > > and B. I have upgraded only the inter.broker.protocol.version to > 0.10.1.0. > > The log.message.format.version is still 0.9.0.1. > > > > I'm writing test data from a java producer to the upgraded cluster A. As > > expected the .timeindex files get created. I understand that the > timestamp > > will be appended to offsets only when I upgrade the > > log.message.format.version. But when I run the following command, I get > the > > following. > > > > kafka-run-class kafka.tools.DumpLogSegments --files > > /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex > > Dumping /mnt/data/kafka-logs/upgrade_test-2/00000000000014500426. > timeindex > > timestamp: 0 offset: 14500426 > > Found timestamp mismatch in > > :/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex > > Index timestamp: 0, log timestamp: -1 > > Index timestamp: 0, log timestamp: -1 > > Found out of order timestamp in > > :/mnt/data/kafka-logs/upgrade_test-2/00000000000014500426.timeindex > > Index timestamp: 0, Previously indexed timestamp: 0 > > > > What does the timestamp mismatch indicate ? is it an error ? > > > > Also the other cluster, Cluster B is mirroring data from cluster A, and > > when the run the dumplogSegment command on that cluster I get something > > like this.. > > > > > > timestamp: 0 offset: 1124701 > > timestamp: 0 offset: 1124701 > > timestamp: 0 offset: 1124701 > > timestamp: 0 offset: 1124701 > > timestamp: 0 offset: 1124701 > > timestamp: 0 offset: 1124701 > > timestamp: 0 offset: 1124701 > > timestamp: 0 offset: 1124701 > > > > Any inputs will be of great help. > > Thanks, > > Meghana > > >
