Hello, We're researching autoscaling our microservices consuming from a Kafka Topic based on the records-lag-max metric. For the following MBean object:
MBeans TAB: kafka.consumer-> consumer-fetcher-manager-metric -> consumer-1 -> Attributes -> records-lag-max In JConsole, we noticed that we consistently get either 1.0 or -Infinity. We also implemented in our Java code to invoke the metrics() method of KafkaConsumer.java to fetch the metrics after a poll of records from the Topic. Map<MetricName,? extends Metric> metrics() //Get the metrics kept by the consumer Printing out this record also shows 1.0 or -Infinity. In our test we have a Kafka Producer sending in about 50 messages/s while a Kafka Consumer is consuming about 20 messages/s. We know that the Kafka Consumer is definitely lagging behind because it continues to consume messages well after shutting down the Kafka Producer. Is this a known bug to get the correct value for records-lag-max or do we need to tune the configuration settings somehow? We're running 0.10.1.1. Thanks for your help.
