One additional note: In FlinkKafkaConsumer 0.9+, the current read offset should already exist in Flink metrics. See https://issues.apache.org/jira/browse/FLINK-4186.
But yes, this is still missing for 0.8, so you need to directly query ZK for this. Cheers, Gordon On 21 April 2017 at 8:28:09 AM, Tzu-Li (Gordon) Tai (tzuli...@apache.org) wrote: Hi Sandeep, It isn’t fixed yet, so I think external tools like the Kafka offset checker still won’t work. If you’re using 08 and is currently stuck with this issue, you can still directly query ZK to get the offsets. I think for FlinkKafkaConsumer09 the offset is exposed to Flink's metric system using Kafka’s own returned metrics, but for 08 this is still missing. There is this JIRA [1] that aims at exposing consumer lag across all Kafka consumer versions to Flink metrics. Perhaps it would make sense to also generally expose the offset for all Kafka consumer versions to Flink metrics as well. - Gordon [1] https://issues.apache.org/jira/browse/FLINK-6109 On 19 April 2017 at 5:11:11 AM, sandeep6 (vr1meghash...@gmail.com) wrote: Is this fixed now? If not, is there any way to monitor kafka offset that is being processed by Flink? This should be a use case for everyone who uses Flink with Kafka. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-Consumer-Behaviour-tp8257p12663.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
