Hey Georg, thanks for your interest. We do have an open ticket for that task: https://issues.apache.org/jira/browse/KAFKA-8432, and we believe it's something worth doing just didn't get chance to implement. Feel free to take it if you want.
Guozhang and Matthias, could you share your opinion on this matter? I couldn't recall the original discussion we had regarding whether to have this add-on feature, or its impact. Boyang ________________________________ From: Georg Friedrich <georg.friedr...@webfleet.com> Sent: Tuesday, October 8, 2019 6:26 AM To: dev@kafka.apache.org <dev@kafka.apache.org> Subject: RE: group.instance.id for StickyAssignors Hi Matthias, Let's see if I can explain my idea in more detail: As far as I understood the group.instance.id is used to assign some ID to the consumer, so it keeps the same ID even if the consumer is restarted and by that allowing Kafka to keep the old assignment order. This is why it got part e.g. of the RangeAssignor, so that a restart of a consumer always ends up in the same partition assignment. The StickyAssignors in turn tend to keep the assignments as good as possible, to reduce the amount of moving partitions while doing a rebalance. But there is the one situation where you have unassigned partitions and currently the AbstractStickyAssignor class will just grab the very first consumer from a set of consumers that has the lowest amount of subscriptions (see https://github.com/axbaretto/kafka/blob/master/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java#L387). That said you can't predict which consumer will be used on the first assignment as the generated consumer member IDs are just compared in an ordinary fashion (see https://github.com/axbaretto/kafka/blob/master/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java#L666). I was wondering whether it is planned to use the group.instance.id there (in the same way as it was done for the RangeAssignor), so that the first assignment takes note of the group.instance.id. Please correct me, if I got something wrong. Thanks in advance for your help. Kind regards Georg Friedrich -----Original Message----- From: Matthias J. Sax Sent: Monday, October 7, 2019 2:34 PM To: dev@kafka.apache.org Subject: Re: group.instance.id for StickyAssignors `groud.instance.id` is for static group membership. What advantage to you imply by re-using it for (Cooperative)StickyAssignor? >> As far as I understand this would make the behavior of those assignors more >> predictable. Can you elaborate? I am not aware of any re-use, and I personally believe, that config parameters should have a single purpose and should not be overloaded. Also note, if you set `group.instance.id` you opt-in to (ie, enable) static group membership. Hence, if your intent it to use it for regular re-balancing, that won't work. What issue do you see with current (Cooperative)StickyAssignor that you try to address? -Matthias On 10/7/19 3:13 AM, Georg Friedrich wrote: > Hi, > > I'm wondering whether it is planned to make use of the group.instance.id also > in the (Cooperative)StickyAssignor. > As far as I understand this would make the behavior of those assignors more > predictable. > Maybe you have some information on this or whether I should open a > Jira-Ticket for it. > Thanks in advance! > > Kind regards, > Georg Friedrich >
