Re: Kafka offset moved back on rebalance

2016-03-19 Thread Daniel Fanjul
Hi, The offsets could belong to different partitions of the same topic. Each partition has its own offset. Try logging *record.partition()*. Regards, Daniel. On Thu, Mar 17, 2016 at 1:58 PM, Erik Pettersson wrote: > Hi, > > I have the 3 consumers subscribing from a single topic with the foll

consumer too fast

2016-03-31 Thread Daniel Fanjul
Hi all, My problem: If the consumer fetches too much data and the processing of the records is not fast enough, commit() fails because there was a rebalance. I cannot reduce 'max.partition.fetch.bytes' because there might be large messages. I don't want to increase the 'session.timeout.ms', beca

Re: consumer too fast

2016-03-31 Thread Daniel Fanjul
vvhuBZO/low-latency-high-message-size-variance > > Example code is here > > https://github.com/omkreddy/kafka-examples/blob/master/consumer/src/main/java/kafka/examples/consumer/advanced/AdvancedConsumer.java > > On Thu, Mar 31, 2016 at 3:43 PM, Daniel Fanjul < > daniel.fan