Ah, thank you - so there is in fact a ticket. That's all I wanted to know. Thanks guys. See you over there...
Kind regards Georg Friedrich -----Original Message----- From: Matthias J. Sax Sent: Monday, October 7, 2019 3:48 PM To: dev@kafka.apache.org Subject: Re: group.instance.id for StickyAssignors As we have a ticket, I would recommend to move the discussion to the ticket. -Matthias On 10/7/19 3:40 PM, Boyang Chen wrote: > Hey Georg, > > thanks for your interest. We do have an open ticket for that task: > https://issues.apache.org/jira/browse/KAFKA-8432, and we believe it's > something worth doing just didn't get chance to implement. Feel free to take > it if you want. > > Guozhang and Matthias, could you share your opinion on this matter? I > couldn't recall the original discussion we had regarding whether to have this > add-on feature, or its impact. > > Boyang > > ________________________________ > From: Georg Friedrich > Sent: Tuesday, October 8, 2019 6:26 AM > To: dev@kafka.apache.org <dev@kafka.apache.org> > Subject: RE: group.instance.id for StickyAssignors > > Hi Matthias, > > Let's see if I can explain my idea in more detail: > As far as I understood the group.instance.id is used to assign some ID to the > consumer, so it keeps the same ID even if the consumer is restarted and by > that allowing Kafka to keep the old assignment order. > This is why it got part e.g. of the RangeAssignor, so that a restart of a > consumer always ends up in the same partition assignment. > > The StickyAssignors in turn tend to keep the assignments as good as possible, > to reduce the amount of moving partitions while doing a rebalance. > But there is the one situation where you have unassigned partitions and > currently the AbstractStickyAssignor class will just grab the very first > consumer from a set of consumers that has the lowest amount of subscriptions > (see > https://github.com/axbaretto/kafka/blob/master/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java#L387). > That said you can't predict which consumer will be used on the first > assignment as the generated consumer member IDs are just compared in an > ordinary fashion (see > https://github.com/axbaretto/kafka/blob/master/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java#L666). > I was wondering whether it is planned to use the group.instance.id there (in > the same way as it was done for the RangeAssignor), so that the first > assignment takes note of the group.instance.id. > > Please correct me, if I got something wrong. > Thanks in advance for your help. > > Kind regards > Georg Friedrich > > -----Original Message----- > From: Matthias J. Sax > Sent: Monday, October 7, 2019 2:34 PM > To: dev@kafka.apache.org > Subject: Re: group.instance.id for StickyAssignors > > `groud.instance.id` is for static group membership. > > What advantage to you imply by re-using it for (Cooperative)StickyAssignor? > >>> As far as I understand this would make the behavior of those assignors more >>> predictable. > > Can you elaborate? > > I am not aware of any re-use, and I personally believe, that config > parameters should have a single purpose and should not be overloaded. > Also note, if you set `group.instance.id` you opt-in to (ie, enable) static > group membership. Hence, if your intent it to use it for regular > re-balancing, that won't work. > > What issue do you see with current (Cooperative)StickyAssignor that you try > to address? > > > -Matthias > > On 10/7/19 3:13 AM, Georg Friedrich wrote: >> Hi, >> >> I'm wondering whether it is planned to make use of the group.instance.id >> also in the (Cooperative)StickyAssignor. >> As far as I understand this would make the behavior of those assignors more >> predictable. >> Maybe you have some information on this or whether I should open a >> Jira-Ticket for it. >> Thanks in advance! >> >> Kind regards, >> Georg Friedrich >> > >
