Dear all, The main purpose is to allow consumers to leave a group permanently, even if they have a static member ID. Additionally, we don't have insights into the use case where a dynamic member might not want to leave the group.
Therefore, should we enhance the close option to support our goal—allowing static members to leave the group when closing the consumer? This means the flag should be renamed to 'releaseStaticMember. Best, Chia-Ping On 2024/09/25 03:22:17 TengYao Chi wrote: > Hi Kirk, > > Thanks for your feedback and questions! > > KT1: > While it might seem that there’s no immediate benefit for non-Kafka Streams > applications to set `leaveGroup=false`, we still offer this option for > consistency with `KafkaStreams#CloseOptions` and to provide flexibility for > future features. However, I agree with your concern that users might misuse > this option, leading to potential throughput issues. To avoid this, we > could include documentation that strongly recommends setting > `leaveGroup=true` unless the user fully understands the implications of > leaving it as `false`. > > KT2: > Thanks for the reminder. > I have updated the KIP to include the actual API changes. Please take a > look . > > KT3: > I believe we should respect the timeout setting, as this offers several key > advantages. First, as Chia-Ping mentioned, it gives users more control over > the close process, allowing them to better manage their application flow. > Second, as a public interface, users will expect the timeout parameter to > act as a contract between Kafka and the user. If the consumer does not > respect this setting, it could lead to confusion and force users to > implement unnecessary workarounds. Ensuring that the `close()` operation > honors the provided timeout will lead to a more predictable and > user-friendly API. > > Thanks again for your questions. > Let me know if you have any further comments. > > Best, > TengYao > > Chia-Ping Tsai <chia7...@gmail.com> 於 2024年9月25日 週三 上午4:51寫道： > > > hi Kirk > > > > > KT1: Why would a non-Kafka Streams application want to set > > leaveGroup=false? Because Kafka Streams manages the group membership > > assignment under the covers, it can re-assign partitions to a new Consumer > > when the old one closes. But in a non-Kafka Streams application, doesn’t > > this just leave the partitions assigned until the coordinator kicks the > > member out of the group? > > > > That's definitely a great question. I don’t have a clear answer regarding > > whether leaveGroup=false means not sending a LEAVE_GROUP request, but I > > have a crazy idea—what if leaveGroup=false means temporarily leaving the > > group? My idea is that we could potentially integrate dynamic members with > > static members. > > > > Here’s how it could work with changes to the new protocol and consumer: > > > > 1. The new consumer will exclusively use static members. If a user > > doesn't specify one, the new consumer will automatically generate it > > (similar to KIP-1082). > > 2. Users can define a close option to either leave the group permanently > > or temporarily. > > > > This approach offers several advantages: > > > > 1. It addresses the needs of Kafka Streams (if they start to use new > > consumer) > > 2. It simplifies the new coordinator, protocol, and consumer, as they no > > longer need to handle two types of members. > > 3. It addresses KIP-1082. The new consumer always sends a LEAVE_GROUP > > request during closing, even if it hasn't received a response from the > > server. > > > > > KT3: Does setting leaveGroup=true carry the *guarantee* that the member > > will leave the group? I’m currently battling some edge cases where close() > > is called with a zero timeout and it times out before the consumer can > > leave the group cleanly. > > > > In my opinion, this is a trade-off between 'not honoring the timeout' and > > 'creating ghost members.' However, I prefer honoring the timeout, as it > > provides users with more control over the 'close' process. We should trust > > that users fully understand the options they choose. > > > > > > Best, > > Chia-Ping > > > > > > > > Kirk True <k...@kirktrue.pro> 於 2024年9月24日 週二 上午8:37寫道： > > > > > Hi TengYao, > > > > > > Thanks for writing up this KIP :) > > > > > > Questions: > > > > > > KT1: Why would a non-Kafka Streams application want to set > > > leaveGroup=false? Because Kafka Streams manages the group membership > > > assignment under the covers, it can re-assign partitions to a new > > Consumer > > > when the old one closes. But in a non-Kafka Streams application, doesn’t > > > this just leave the partitions assigned until the coordinator kicks the > > > member out of the group? > > > > > > KT2: Can you add the actual API changes to the KIP? > > > > > > KT3: Does setting leaveGroup=true carry the *guarantee* that the member > > > will leave the group? I’m currently battling some edge cases where > > close() > > > is called with a zero timeout and it times out before the consumer can > > > leave the group cleanly. > > > > > > Thanks, > > > Kirk > > > > > > > On Sep 22, 2024, at 9:26 PM, TengYao Chi <kiting...@gmail.com> wrote: > > > > > > > > Hello everyone, > > > > > > > > I would like to start a discussion thread on KIP-1092 > > > > <https://cwiki.apache.org/confluence/x/JQstEw>, which proposes > > extending > > > > Consumer#close with an option to leave the consumer group or not. > > > > > > > > Please take a look and let me know what you think, and I would > > appreciate > > > > any suggestions and feedback. > > > > > > > > Best regards, > > > > TengYao > > > > > > > > >
