Thank you for your replies Gwen and Jason. Lets continue the discussion in the JIRA.
On Fri, Jan 22, 2016 at 8:37 PM, Jason Gustafson <ja...@confluent.io> wrote: > The offset API is one of the known gaps in the new consumer. There is a > JIRA (KAFKA-1332), but we might need a KIP to make that change now that the > API is released. For now, Gwen's suggestion is the only way to do it. > > -Jason > > On Thu, Jan 21, 2016 at 8:22 PM, Gwen Shapira <g...@confluent.io> wrote: > > > Hi Robert! > > > > Jason is the expert, and I hope he'll respond soon. > > > > Meanwhile: I think that you can do what you are trying to do by: > > 1. call position() to get the current position you are consuming > > 2. call seekToEnd() and then position(), which will give you the last > > position at the point in which you called seekToEnd() > > 3. Use another seek() call to go to the offset you really want to get to. > > > > I don't think there are current plans to add getOffsetsBefore, but maybe > we > > need it for the use-case you specified. > > I think the developer mailing list (or a JIRA) will be a better place for > > an API discussion. > > > > Gwen > > > > On Wed, Jan 20, 2016 at 3:09 AM, Robert Metzger <rmetz...@apache.org> > > wrote: > > > > > Hi, > > > > > > I'm currently looking into implementing a load shedding strategy into > > > Flink's Kafka consumer. > > > > > > Therefore, I would like to allow users to request the latest offset of > > the > > > subscribed TopicPartitions, so that they can > > > a) determine the lag > > > b) maybe set the next fetch offset to the latest offset (or anything in > > > between) > > > > > > With the SimpleConsumer, there is the getOffsetsBefore() method, which > > > allows to request the largest offset. > > > For the new KafkaConsumer, I only found the seekToEnd() and > > > seekToBeginning() methods, but they don't allow to get the actual end > or > > > beginning. > > > > > > > > > Are there plans to add such an API to the KafkaConsumer again? > > > Can you suggest any workarounds? > > > > > > Regards, > > > Robert > > > > > >
