This is not the case in my situation. I am using an older version where it doesn't have the subscriber model, and processing messages can occur concurrently in multiple threads.
If I use the KafkaConsumer with multiple threads and partitions - kafkaConsumer.run(5) //5 thread count - How can I commit an offset for a specific thread and partition in kafka-clients 0.9.0.1? Thanks Regards, Amir Zuker On Tue, Aug 16, 2016 at 1:36 PM, Sudev A C <sudev...@goibibo.com> wrote: > Hi, > > Message object consists of partition, topic, offset and message. > https://kafka.apache.org/090/javadoc/index.html?org/apache/ > kafka/clients/consumer/KafkaConsumer.html > You many use this to get current offsets for topic-partition combination. > > Thanks > Sudev > > On Tue, Aug 16, 2016 at 2:20 PM, Amir Zuker <a.zu...@gmail.com> wrote: > > > Hi everyone, > > > > I have a question regarding the 'KafkaConsumer' and its API in regards to > > committing offsets. (kafka-clients 0.9.0.1) > > > > *The scenario -* > > I am working with auto commit set to disabled because I want to > implement a > > retry mechanism and eventually transfer the message to another topic that > > contains the poison messages. > > Since I want it to be reliable, I am not using the auto commit and I wish > > to take control on when that should happen > > > > *The implementation detail -* > > My class that extends 'Runnable' and is created by the KafkaConsumer > needs > > to commit the offset once it is done with handling the topic message. > > However, the API for committing messages is located on the KafkaConsumer > > with no relation to partition or thread. > > > > *The problem -* > > If I understand correctly, I can use the same KafkaConsumer instance with > > multiple threads against multiple partitions. > > If that is the case, how can I commit the offset specific to my > 'Runnable' > > instance that just processed a single message without affecting other > > threads and partitions? > > > > Thanks in advance, > > Amir Zuker > > > > > > -- > Thanks > Sudev A C > Data Team >
