Re: doubt regarding High level consumer commiting offset

2014-08-22 Thread Guozhang Wang
Hello Arjun, Currently there is no easy way to re-consume the message that has already been returned to you that caused the exception; what you need to do is to close the current consumer in handling the exception WITHOUT committing the offset, fix the issue that caused the exception, and then res

Re: doubt regarding High level consumer commiting offset

2014-08-22 Thread Arjun
The way i am doing this is as below First i get a map of topic to stream list Map>> consumerMap = consumer.createMessageStreams(topicCountMap); From that i get the Stream list for my topic List> stream = consumerMap.get(topic); I iterate on the list, for each stream i create a new

doubt regarding High level consumer commiting offset

2014-08-22 Thread Arjun
Hi, I have My Kafka setup with 0.8.1 kafka with 3 producers and 3 consumers. I use the high level consumer. My doubt is , i read the messages from the consumer iterator, and after reading the message i process those messages, if the processing throws any exception i am not commiting any offse