This is a bug in the tool. Please file a bug and attach these error/info logs to it.
Thanks, Neha On Thu, Jun 26, 2014 at 5:24 AM, Yury Ruchin <yuri.ruc...@gmail.com> wrote: > I have set log level to DEBUG and saw something strange in the output. For > each topic partition, I see the following pattern: > > [2014-06-26 16:00:24,467] ERROR No owner for partition [<topic_name>,0] > (kafka.tools.VerifyConsumerRebalance$) > > ... > > [2014-06-26 16:00:24,469] INFO Owner of partition [<topic_name>,0] is > <consumer_name>_<node_name>-1403049600000-abc12345-0 > (kafka.tools.VerifyConsumerRebalance$) > > As I understand VerifyConsumerRebalance.scala, those 2 messages should be > mutually exclusive, but their both appear for every partition. With the > default log settings, only that with ERROR level is shown to user. > > Is this a problem with the tool? > > 2014-06-25 2:04 GMT+04:00 Neha Narkhede <neha.narkh...@gmail.com>: > > > I would turn on DEBUG on the tool to see which url it reads and doesn't > > find the owners. > > > > > > > > > > On Tue, Jun 24, 2014 at 11:28 AM, Yury Ruchin <yuri.ruc...@gmail.com> > > wrote: > > > > > I've just double-checked. The URL is correct, the same one is used by > > Kafka > > > clients. > > > > > > > > > 2014-06-24 22:21 GMT+04:00 Neha Narkhede <neha.narkh...@gmail.com>: > > > > > > > Is it possible that maybe the zookeeper url used for the > > > > VerifyConsumerRebalance tool is incorrect? > > > > > > > > > > > > On Tue, Jun 24, 2014 at 12:02 AM, Yury Ruchin <yuri.ruc...@gmail.com > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > I've run into the following problem. I try to read from a > > 50-partition > > > > > Kafka topic using high level consumer with 8 streams. I'm using > > > 8-thread > > > > > pool, each thread handling one stream. After a short time, the > > threads > > > > > reading from the stream stop reading. Lag between topic latest > offset > > > and > > > > > the consumer constantly increases as new messages come in. > > > > > > > > > > I looked into ZK: /consumers/<consumer_name>/owners/<topic_name> > and > > > > see a > > > > > list of znodes corresponding to the full list of partitions: [1, 2, > > 3, > > > > > ...]. When I do zk get on e. g. > > > > > /consumers/<consumer_name>/owners/<topic_name>/1 - I see a valid > > > consumer > > > > > name corresponding to Kafka client logs, e. g. > > > > > <consumer_name>_<node_name>-1403049600000-abc12345-0. However, > when I > > > run > > > > > the VerifyConsumerRebalance tool, I see the following: > > > > > > > > > > No owner for partition [<topic_name>,1] > > > > > (kafka.tools.VerifyConsumerRebalance$) > > > > > > > > > > No owner for partition [<topic_name>,2] > > > > > (kafka.tools.VerifyConsumerRebalance$) > > > > > > > > > > ... > > > > > > > > > > No owner for partition [<topic_name>,50] > > > > > (kafka.tools.VerifyConsumerRebalance$) > > > > > > > > > > According to this output, no partition has owner, which seemingly > > > > > contradicts to what I see in ZK. > > > > > > > > > > What would cause such a problem and how can I troubleshoot it > > further? > > > > > > > > > > Thanks! > > > > > > > > > > > > > > >
