Static-Group membership ships with AK 2.3 (the open tickets of the KIP
are minor):

https://cwiki.apache.org/confluence/display/KAFKA/KIP-345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances

There is also KIP-415 for Kafka Connect in AK 2.3:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-415%3A+Incremental+Cooperative+Rebalancing+in+Kafka+Connect



Currently WIP is KIP-429 and KIP-441:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-429%3A+Kafka+Consumer+Incremental+Rebalance+Protocol

https://cwiki.apache.org/confluence/display/KAFKA/KIP-441%3A+Smooth+Scaling+Out+for+Kafka+Streams



On 7/19/19 12:31 PM, Jeff Widman wrote:
> I am also interested in learning how others are handling this.
> 
> I also support several services where average message processing time takes
> 20 seconds per message but p99 time is about 20 minutes and the
> stop-the-world rebalancing is very painful
> 
> On Fri, Jul 19, 2019, 11:38 AM Raman Gupta <rocketra...@gmail.com> wrote:
> 
>> I've found
>> https://cwiki.apache.org/confluence/display/KAFKA/Incremental+Cooperative+Rebalancing:+Support+and+Policies
>> and
>> https://cwiki.apache.org/confluence/display/KAFKA/Incremental+Cooperative+Rebalancing+for+Streams
>> .
>> This is *exactly* what I need, right down to the Kubernetes pod
>> restart case. The number of issues with the current approach to
>> rebalancing elucidated in these documents is downright scary, and now
>> I am not surprised I am having tonnes of issues.
>>
>> Are there any plans to start implementing delayed imbalance and
>> standby bootstrap?
>>
>> Are there any short-term best practices that can help alleviate these
>> issues? My main problem right now is the "Instance Bounce" and
>> "Instance Failover" scenarios, and according to this wiki page,
>> num.standby.replicas should help with at least the former. Can someone
>> explain what this does?
>>
>> Regards,
>> Raman
>>
>> On Fri, Jul 19, 2019 at 12:53 PM Raman Gupta <rocketra...@gmail.com>
>> wrote:
>>>
>>> I have a situation in which the current rebalancing algorithm seems to
>>> be extremely sub-optimal.
>>>
>>> I have a topic with 100 partitions, and up to 100 separate consumers.
>>> Processing each message on this topic takes between 1 and 20 minutes,
>>> depending on the message.
>>>
>>> If any of the 100 consumers dies or drops out of the group, there is a
>>> huge amount of idle time as many consumers (up to 99 of them) finish
>>> their work and sit around idle, just waiting for the rebalance to
>>> complete.
>>>
>>> In addition, with 100 consumers, its not unusual for one to die for
>>> one reason or another, so these stop-the-world rebalances are
>>> happening all the time, making the entire system slow to a snail's
>>> pace.
>>>
>>> It surprises me that rebalance is so inefficient. I would have thought
>>> that partitions would just be assigned/unassigned to consumers in
>>> real-time without waiting for the entire consumer group to quiesce.
>>>
>>> Is there anything I can do to improve matters?
>>>
>>> Regards,
>>> Raman
>>
> 

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to