[A] Unfortunately, we only documented this in the code: /** * For verifying the consistency among replicas. * * 1. start a fetcher on every broker. * 2. each fetcher does the following * 2.1 issues fetch request * 2.2 puts the fetched result in a shared buffer * 2.3 waits for all other fetchers to finish step 2.2 * 2.4 one of the fetchers verifies the consistency of fetched results among replicas * * The consistency verification is up to the high watermark. The tool reports the * max lag between the verified offset and the high watermark among all partitions. * * If a broker goes down, the verification of the partitions on that broker is delayed * until the broker is up again. * * Caveats: * 1. The tools needs all brokers to be up at startup time. * 2. The tool doesn't handle out of range offsets. *
Feel free to contribute an improvement to our official documentation. [B] Apparently this is the expected behavior if the consumer didn't see new messages for 10 seconds. On Thu, Jun 9, 2016 at 7:05 AM, Kamal C <kamaltar...@gmail.com> wrote: > Hi, > > A. What is the usage of `kafka-replica-verification.sh` script ? I don't > find any documentations about it in [1] and [2]. > > I have a topic `test` with 10 partitions. Ran the above script, it > continuously prints the below results. > > [kamal@tcltest1 bin]$ sh kafka-replica-verification.sh --time -2 > --topic-white-list test --report-interval-ms 1000 --broker-list > localhost:9092 > 2016-06-07 12:49:13,664: verification process is started. > 2016-06-07 12:49:14,632: max lag is 0 for partition [test,6] at offset > 39008 among 10 partitions > 2016-06-07 12:49:15,633: max lag is 0 for partition [test,6] at offset > 39008 among 10 partitions > 2016-06-07 12:49:16,634: max lag is 0 for partition [test,6] at offset > 39008 among 10 partitions > 2016-06-07 12:49:17,635: max lag is 0 for partition [test,6] at offset > 39008 among 10 partitions > 2016-06-07 12:49:18,636: max lag is 0 for partition [test,6] at offset > 39008 among 10 partitions > 2016-06-07 12:49:19,637: max lag is 0 for partition [test,6] at offset > 39008 among 10 partitions > > B. `kafka-replay-log-producer.sh` script throws ConsumerTimeoutException. > What is wrong with the below command ? > > [kamal@tcltest1 bin]$ sh kafka-replay-log-producer.sh --broker-list > localhost:9092 --inputtopic test --outputtopic xyz --reporting-interval > 1000 --zookeeper localhost:2181 > [2016-06-07 12:42:10,925] ERROR consumer thread timing out > (kafka.tools.ReplayLogProducer$ZKConsumerThread) > kafka.consumer.ConsumerTimeoutException > at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69) > at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) > at > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:64) > at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:56) > at scala.collection.Iterator$class.foreach(Iterator.scala:893) > at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > at kafka.consumer.KafkaStream.foreach(KafkaStream.scala:25) > at > kafka.tools.ReplayLogProducer$ZKConsumerThread.run(ReplayLogProducer.scala:140) > > > [1]: http://kafka.apache.org/documentation.html > [2]: https://cwiki.apache.org/confluence/display/KAFKA/System+Tools
