Hi Robert! Jason is the expert, and I hope he'll respond soon.
Meanwhile: I think that you can do what you are trying to do by: 1. call position() to get the current position you are consuming 2. call seekToEnd() and then position(), which will give you the last position at the point in which you called seekToEnd() 3. Use another seek() call to go to the offset you really want to get to. I don't think there are current plans to add getOffsetsBefore, but maybe we need it for the use-case you specified. I think the developer mailing list (or a JIRA) will be a better place for an API discussion. Gwen On Wed, Jan 20, 2016 at 3:09 AM, Robert Metzger <rmetz...@apache.org> wrote: > Hi, > > I'm currently looking into implementing a load shedding strategy into > Flink's Kafka consumer. > > Therefore, I would like to allow users to request the latest offset of the > subscribed TopicPartitions, so that they can > a) determine the lag > b) maybe set the next fetch offset to the latest offset (or anything in > between) > > With the SimpleConsumer, there is the getOffsetsBefore() method, which > allows to request the largest offset. > For the new KafkaConsumer, I only found the seekToEnd() and > seekToBeginning() methods, but they don't allow to get the actual end or > beginning. > > > Are there plans to add such an API to the KafkaConsumer again? > Can you suggest any workarounds? > > Regards, > Robert >
