hi Kirk > KT1: Why would a non-Kafka Streams application want to set leaveGroup=false? Because Kafka Streams manages the group membership assignment under the covers, it can re-assign partitions to a new Consumer when the old one closes. But in a non-Kafka Streams application, doesn’t this just leave the partitions assigned until the coordinator kicks the member out of the group?
That's definitely a great question. I don’t have a clear answer regarding whether leaveGroup=false means not sending a LEAVE_GROUP request, but I have a crazy idea—what if leaveGroup=false means temporarily leaving the group? My idea is that we could potentially integrate dynamic members with static members. Here’s how it could work with changes to the new protocol and consumer: 1. The new consumer will exclusively use static members. If a user doesn't specify one, the new consumer will automatically generate it (similar to KIP-1082). 2. Users can define a close option to either leave the group permanently or temporarily. This approach offers several advantages: 1. It addresses the needs of Kafka Streams (if they start to use new consumer) 2. It simplifies the new coordinator, protocol, and consumer, as they no longer need to handle two types of members. 3. It addresses KIP-1082. The new consumer always sends a LEAVE_GROUP request during closing, even if it hasn't received a response from the server. > KT3: Does setting leaveGroup=true carry the *guarantee* that the member will leave the group? I’m currently battling some edge cases where close() is called with a zero timeout and it times out before the consumer can leave the group cleanly. In my opinion, this is a trade-off between 'not honoring the timeout' and 'creating ghost members.' However, I prefer honoring the timeout, as it provides users with more control over the 'close' process. We should trust that users fully understand the options they choose. Best, Chia-Ping Kirk True <k...@kirktrue.pro> 於 2024年9月24日 週二 上午8:37寫道： > Hi TengYao, > > Thanks for writing up this KIP :) > > Questions: > > KT1: Why would a non-Kafka Streams application want to set > leaveGroup=false? Because Kafka Streams manages the group membership > assignment under the covers, it can re-assign partitions to a new Consumer > when the old one closes. But in a non-Kafka Streams application, doesn’t > this just leave the partitions assigned until the coordinator kicks the > member out of the group? > > KT2: Can you add the actual API changes to the KIP? > > KT3: Does setting leaveGroup=true carry the *guarantee* that the member > will leave the group? I’m currently battling some edge cases where close() > is called with a zero timeout and it times out before the consumer can > leave the group cleanly. > > Thanks, > Kirk > > > On Sep 22, 2024, at 9:26 PM, TengYao Chi <kiting...@gmail.com> wrote: > > > > Hello everyone, > > > > I would like to start a discussion thread on KIP-1092 > > <https://cwiki.apache.org/confluence/x/JQstEw>, which proposes extending > > Consumer#close with an option to leave the consumer group or not. > > > > Please take a look and let me know what you think, and I would appreciate > > any suggestions and feedback. > > > > Best regards, > > TengYao > >
