Re: MaxLag Mbean for Kafka Consumer

2014-07-26 Thread ravi singh
Thanks Todd!! You were correct about the zookeeper connection loss. On Sun, Jul 27, 2014 at 1:15 AM, Todd Palino wrote: > The MaxLag mbean is only valid for an active consumer. So while the > consumer is actively running, it should be accurate (as far as telling you > the lag on the partition t

Re: MaxLag Mbean for Kafka Consumer

2014-07-26 Thread Todd Palino
The MaxLag mbean is only valid for an active consumer. So while the consumer is actively running, it should be accurate (as far as telling you the lag on the partition that you are consuming that has the most lag). However, if you look at the output from the ConsumerOffsetChecker, you see that the

MaxLag Mbean for Kafka Consumer

2014-07-26 Thread ravi singh
The Max lag Mbean is defined as "Number of messages the consumer lags behind the producer". Now when I read the Mbean value it give me the count as 0 (and occasionally some value like 130 or 340 ) ConsumerFetcherManager.test-consumer-group-MaxLag count = 0 But when I use the kafka.tools.Co

Re: Partitions per Machine for a topic

2014-07-26 Thread Sharninder
You can add partitions at runtime but not delete (AFAIK, someone correct me if I'm wrong). The reason it's usually discouraged is that when partitions get added to a topic, the message assignment changes and if you're application depends on some partitioning logic, that will fail. -- Sharninder