HI:
 We are seeing unexpected rebalances in golang consumers, described below.
        1. We have a topic with 36 partitions,and one consumer (lets name it 
consumer1) consuming it. 
        2. Run kafka  in Docker and configuration:We use defaults  
        3. Consumer  consuming data is slowly about  cost 1s for  one piece of 
data 
        4. All the consumers for topic A are in the same group 
       5. The rebalances are intermittent and hard to reproduce.  We see no 
obvious errors in the logs.
        6.  No matter how to change  the configuration that affects rebalance 
,it always rebalance
                The configuration that affects rebalance below:
 max.poll.interval.ms
                max.poll.records
                request.timeout.ms
                session.timeout.ms 
       As far as I am concerned,  conditions of  the rebalance contains:
          a consumer is considered DEAD by the group coordinator.
              A.  when the consumer is busy, which means that no heartbeats has 
been sent in the meanwhile by the consumer to the group coordinator within the 
configured session interval
              B.  when the consumer is slowly with a long-running processing, 
which means that  interval of  poll() is too long  within the configured 
max.poll.interval.ms。
  question:
       1. is it  correctly the idea about  B ?   i think it maybe the major 
factor for rebalance in my case  ,because  consuming data is slowly.
       2. i have do an experiment  for  assert  B. but not reproduce
         A:consumer fast or slow  can't trigger it
       3. Any tips on APIs or debug techniques to figure out rebalance causes 
  4. How can I trigger it manually ?   
       5. Is it a bad idea to have the same Consumer Group (Same ID) consuming 
from multiple topics ?


   


Reply via email to