We just tried upgrading the jar to 10.2 version and it works! Also, I was misleading with the prior version. We were running 10.0.1 KafkaServer but using the 10.1 client jar. With the 10.2 client jar working against the 10.0.1 KafkaServer, the records-lag-max metrics is now reporting correctly! Anyway, it may be useful to document this for other users the compatibility of this very useful metric. Thanks. From: Benson Chen Sent: Tuesday, May 23, 2017 3:11 PM To: users@kafka.apache.org Cc: Hongjie Xin <hongjie....@sas.com> Subject: MBean for records-lag-max metric always 1.0 or -Infinity
Hello, We're researching autoscaling our microservices consuming from a Kafka Topic based on the records-lag-max metric. For the following MBean object: MBeans TAB: kafka.consumer-> consumer-fetcher-manager-metric -> consumer-1 -> Attributes -> records-lag-max In JConsole, we noticed that we consistently get either 1.0 or -Infinity. We also implemented in our Java code to invoke the metrics() method of KafkaConsumer.java to fetch the metrics after a poll of records from the Topic. Map<MetricName,? extends Metric> metrics() //Get the metrics kept by the consumer Printing out this record also shows 1.0 or -Infinity. In our test we have a Kafka Producer sending in about 50 messages/s while a Kafka Consumer is consuming about 20 messages/s. We know that the Kafka Consumer is definitely lagging behind because it continues to consume messages well after shutting down the Kafka Producer. Is this a known bug to get the correct value for records-lag-max or do we need to tune the configuration settings somehow? We're running 0.10.1.1. Thanks for your help.
