Thanks Todd!! You were correct about the zookeeper connection loss.
On Sun, Jul 27, 2014 at 1:15 AM, Todd Palino <[email protected]> wrote: > The MaxLag mbean is only valid for an active consumer. So while the > consumer is actively running, it should be accurate (as far as telling you > the lag on the partition that you are consuming that has the most lag). > However, if you look at the output from the ConsumerOffsetChecker, you see > that the partition you are looking at (0) has no owner. That means that > there is no consumer in the group that is actively consuming the > partition. That means that MaxLag will not be accurate. > > Now, the other thing that could be going on here is that you could have a > consumer that has lost its Zookeeper connection, but is still consuming > messages. In that case, MaxLag would be correct for that consumer, but > with offsets not being committed, it means that when you stop that > consumer and another one picks up the partition, it will start from the > last committed offset, which means it¹s going to duplicate a lot of > messages. > > -Todd > > > On 7/26/14, 8:03 AM, "ravi singh" <[email protected]> wrote: > > >The Max lag Mbean is defined as "Number of messages the consumer lags > >behind the producer". Now when I read the Mbean value it give me the count > >as 0 (and occasionally some value like 130 or 340 ) > > > >ConsumerFetcherManager.test-consumer-group-MaxLag count = 0 > > > > > >But when I use the kafka.tools.ConsumerOffsetChecker I get following > >as Lag value > > > >*Group Topic Pid Offset > >logSize *Lag* Owner* > >test-consumer-group kafka-test 0 275985215 > >276195685 *210470* none > > > > > >Are both the lags not related? Or is it like I am capturing the lag MaxLag > >incorrectly? > > > >-- > >*Regards,* > >*Ravi* > > -- *Regards,* *Ravi*
