Re: Frequent consumer rebalances, auto commit failures

2018-06-03 Thread Shantanu Deshmukh
Hi, I do not have trace level logs as of now. I am doing very basic operation with messages. The only time consuming part is sending an e-mail. Our Email servers are very slow so sending one email is taking upto 20 seconds. That's why I turned max.poll.records to just 2, keppt session time out at 1

Re: Frequent consumer rebalances, auto commit failures

2018-06-01 Thread Ken Chen
1. Any detail logs ? 2. How do you process the records after you polled the records? 3. How much time does it take for every round of poll ? Thanks ! -- Sent from my iPhone On May 28, 2018, at 10:44 PM, Shantanu Deshmukh wrote: Can anyone here help me please? I am at my wit's end. I now have

Re: Frequent consumer rebalances, auto commit failures

2018-05-28 Thread Shantanu Deshmukh
Can anyone here help me please? I am at my wit's end. I now have max.poll.records set to just 2. Still I am getting Auto offset commit failed warning. Log file is getting full because of this warning. Session timeout is 5 minutes, max.poll.interval.ms is 10 minutes. On Wed, May 23, 2018 at 12:42 P

Fwd: Frequent consumer rebalances, auto commit failures

2018-05-23 Thread Shantanu Deshmukh
Hello, We have a 3 broker Kafka 0.10.0.1 cluster. There we have 3 topics with 10 partitions each. We have an application which spawns threads as consumers. We spawn 5 consumers for each topic. I am observing that consider group randomly keeps rebalancing. Then many times we see logs saying "Revoki

Frequent consumer rebalances, auto commit failures

2018-05-22 Thread Shantanu Deshmukh
We have a 3 broker Kafka 0.10.0.1 cluster. There we have 3 topics with 10 partitions each. We have an application which spawns threads as consumers. We spawn 5 consumers for each topic. I am observing that consider group randomly keeps rebalancing. Then many times we see logs saying "Revoking parti