Hey, Have a look at [1]. Basically, you won't see the "real-time" consumer group offsets stored in Kafka itself, but only the ones the Flink Kafka consumer stores there when checkpointing (assuming you have checkpointing enabled). The same information is available in Flink metrics [2], "committedOffsets". I'm not sure, but I think "currentOffsets" could be the real-time offset, what you're probably looking for.
[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-consumers-offset-committing-behaviour-configuration [2] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/metrics.html#kafka-connectors Regards, Juha El mié, 16 dic 2020 a las 8:22, Rex Fenley (<r...@remind101.com>) escribió: > I'll give a look into that approach. Thanks > > On Tue, Dec 15, 2020 at 9:48 PM Aeden Jameson <aeden.jame...@gmail.com> > wrote: > >> My understanding is the FlinkKafkaConsumer is a wrapper around the >> Kafka consumer libraries so if you've set the group.id property you >> should be able to see the offsets with something like >> kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe >> --group my-flink-application. >> >> On Tue, Dec 15, 2020 at 9:39 PM Rex Fenley <r...@remind101.com> wrote: >> > >> > Hi, >> > >> > Is there any way to fetch the current kafka topic offsets for the kafka >> sources for flink? >> > >> > Thanks! >> > >> > -- >> > >> > Rex Fenley | Software Engineer - Mobile and Backend >> > >> > >> > Remind.com | BLOG | FOLLOW US | LIKE US >> >> >> >> -- >> Cheers, >> Aeden >> >> GitHub: https://github.com/aedenj >> Linked In: http://www.linkedin.com/in/aedenjameson >> Blah Blah Blah: http://www.twitter.com/daliful >> > > > -- > > Rex Fenley | Software Engineer - Mobile and Backend > > > Remind.com <https://www.remind.com/> | BLOG <http://blog.remind.com/> | > FOLLOW US <https://twitter.com/remindhq> | LIKE US > <https://www.facebook.com/remindhq> >
