Re: Very long consumer rebalances

Can anyone help me understand how to debug this issue? I tried setting log level to trace in consumer logback configuration. But at such times nothing appears in log, even in trace level. It is like entire code is frozen. On Thu, Aug 16, 2018 at 6:32 PM Shantanu Deshmukh wrote: > I saw a few top

Re: Very long consumer rebalances

I saw a few topics with segment.ms and retention.ms property set. Can that be causing any issue? I remember that this is the only change I carried out to the cluster in last couple of months after which the problem started. On Fri, Aug 10, 2018 at 2:55 PM M. Manna wrote: > if you can upgrade, I

Re: Very long consumer rebalances

Please remove me from the list On Fri, Jul 6, 2018 at 2:55 AM Shantanu Deshmukh wrote: > Hello everyone, > > We are running a 3 broker Kafka 0.10.0.1 cluster. We have a java app which > spawns many consumer threads consuming from different topics. For every > topic we have specified different co

Re: Very long consumer rebalances

All this is fine. But what I do not understand is, why only some of the consumer groups start very late. We have 7 topics and their consumers belong to 7 different CGs. Whenever I want to restart my application Only one or two of them will start very late. It takes almost 5-10 minutes before their

Re: Very long consumer rebalances

if you can upgrade, I would say upgrading to 0.10.2.x would be better for you (or even higher, 2.0.0). Otherwise you have to play around with max.poll.records and session.timeout.ms. As the doc says (or newer versions), the adjustment should be such that request.timeout.ms >= max.poll.interval.ms.

Re: Very long consumer rebalances

Hi, Yes my consumer application works like below 1. Reads how many workers are required to process each topics from properties file 2. As many threads are spawned as there are workers mentioned in properties file, topic name is passed to this thread. FixedThreadPool implementation

Re: Very long consumer rebalances

In v0.10.0.1, consumer heartbeat background thread feature is not available. Lot of users faced similar errors. So, KIP-62 is proposed. You have to update your Kafka version

Re: Very long consumer rebalances

In the simplest way, how have you implemented your consumer? 1) Does your consumers join a designated group, process messages, and then closes all connection? Or does it stay open perpetually until server shutdown? 2) Have you configured the session timeouts for client and zookeeper accordingly?

Re: Very long consumer rebalances

I am facing too many problems these days. Now one of our consumer groups is rebalancing every now and then. And rebalance takes very low, more than 5-10 minutes. Even after re-balancing I see that only half of the consumers are active/receive assignment. Its all going haywire. I am seeing these l

Re: Very long consumer rebalances

It's a very good and important doc so I think you should read it all. You should get some idea from sections like *Detecting Consumer Failures* and *Multi-threaded Processing* for your case. On Thu, Jul 12, 2018, 3:17 PM Shantanu Deshmukh wrote: > Hi Steve, > > Could you please shed more light

Re: Very long consumer rebalances

Hi Steve, Could you please shed more light on this? What section should I revisit? I am using high-level consumer. So I am simply calling consumer.close() when I am shutting down the process. Is there any other method to be called before calling close()? On Mon, Jul 9, 2018 at 5:58 PM Steve Tian

Re: Very long consumer rebalances

Please re-read the javadoc of KafkaConsumer, make sure you know how to wakeup/close consumer properly while shutting down your application. Try to understand the motivation of KIP-62 and adjust related timeout. On Mon, Jul 9, 2018, 8:05 PM harish lohar wrote: > Try reducing below timer > metada

Re: Very long consumer rebalances

Try reducing below timer metadata.max.age.ms = 30 On Fri, Jul 6, 2018 at 5:55 AM Shantanu Deshmukh wrote: > Hello everyone, > > We are running a 3 broker Kafka 0.10.0.1 cluster. We have a java app which > spawns many consumer threads consuming from different topics. For every > topic we hav

Re: Very long consumer rebalances

Kind people on this group, please help me! On Fri, Jul 6, 2018 at 3:24 PM Shantanu Deshmukh wrote: > Hello everyone, > > We are running a 3 broker Kafka 0.10.0.1 cluster. We have a java app which > spawns many consumer threads consuming from different topics. For every > topic we have specified

Very long consumer rebalances

Hello everyone, We are running a 3 broker Kafka 0.10.0.1 cluster. We have a java app which spawns many consumer threads consuming from different topics. For every topic we have specified different consumer-group. A lot of times I see that whenever this application is restarted a CG on one or two t