Since it is something special to Kafka 0.8, it could be implemented in a simple addition to the ZooKeeperOffsetHandler used by the FlinkKafkaConsumer08.
Would you be willing to contribute this? That would certainly help speeding up the resolution of the issue... On Fri, Apr 21, 2017 at 2:33 AM, Tzu-Li (Gordon) Tai <tzuli...@apache.org> wrote: > One additional note: > > In FlinkKafkaConsumer 0.9+, the current read offset should already exist > in Flink metrics. > See https://issues.apache.org/jira/browse/FLINK-4186. > > But yes, this is still missing for 0.8, so you need to directly query ZK > for this. > > Cheers, > Gordon > > > On 21 April 2017 at 8:28:09 AM, Tzu-Li (Gordon) Tai (tzuli...@apache.org) > wrote: > > Hi Sandeep, > > It isn’t fixed yet, so I think external tools like the Kafka offset > checker still won’t work. > If you’re using 08 and is currently stuck with this issue, you can still > directly query ZK to get the offsets. > > I think for FlinkKafkaConsumer09 the offset is exposed to Flink's metric > system using Kafka’s own returned metrics, but for 08 this is still missing. > > There is this JIRA [1] that aims at exposing consumer lag across all Kafka > consumer versions to Flink metrics. Perhaps it would make sense to also > generally expose the offset for all Kafka consumer versions to Flink > metrics as well. > > - Gordon > > [1] https://issues.apache.org/jira/browse/FLINK-6109 > > > On 19 April 2017 at 5:11:11 AM, sandeep6 (vr1meghash...@gmail.com) wrote: > > Is this fixed now? If not, is there any way to monitor kafka offset that is > being processed by Flink? This should be a use case for everyone who uses > Flink with Kafka. > > > > -- > View this message in context: http://apache-flink-user- > mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-Consumer-Behaviour- > tp8257p12663.html > Sent from the Apache Flink User Mailing List archive. mailing list archive > at Nabble.com. > >
