Hi all, I see that leaveGroup has been renamed as releaseStaticMember. I hate to ask, but what is the behavior when a user sets this to true for a non-static member? Should it just work, log a warning, throw an error, or?
I think I’m actually OK with leaving it as leaveGroup with a lot of documentation that warns users away from changing it arbitrarily. Thanks! > On Sep 26, 2024, at 7:32 AM, TengYao Chi <kiting...@gmail.com> wrote: > > Hi Chia-Ping > > I got your point and updated the content of KIP accordingly. > Please take a look and let me know what you think. > > Best, > TengYao > > Chia-Ping Tsai <chia7...@apache.org> 於 2024年9月26日 週四 下午8:41寫道： > >> Dear all, >> >> The main purpose is to allow consumers to leave a group permanently, even >> if they have a static member ID. Additionally, we don't have insights into >> the use case where a dynamic member might not want to leave the group. >> >> Therefore, should we enhance the close option to support our goal—allowing >> static members to leave the group when closing the consumer? This means the >> flag should be renamed to 'releaseStaticMember. >> >> Best, >> Chia-Ping >> >> On 2024/09/25 03:22:17 TengYao Chi wrote: >>> Hi Kirk, >>> >>> Thanks for your feedback and questions! >>> >>> KT1: >>> While it might seem that there’s no immediate benefit for non-Kafka >> Streams >>> applications to set `leaveGroup=false`, we still offer this option for >>> consistency with `KafkaStreams#CloseOptions` and to provide flexibility >> for >>> future features. However, I agree with your concern that users might >> misuse >>> this option, leading to potential throughput issues. To avoid this, we >>> could include documentation that strongly recommends setting >>> `leaveGroup=true` unless the user fully understands the implications of >>> leaving it as `false`. >>> >>> KT2: >>> Thanks for the reminder. >>> I have updated the KIP to include the actual API changes. Please take a >>> look . >>> >>> KT3: >>> I believe we should respect the timeout setting, as this offers several >> key >>> advantages. First, as Chia-Ping mentioned, it gives users more control >> over >>> the close process, allowing them to better manage their application flow. >>> Second, as a public interface, users will expect the timeout parameter to >>> act as a contract between Kafka and the user. If the consumer does not >>> respect this setting, it could lead to confusion and force users to >>> implement unnecessary workarounds. Ensuring that the `close()` operation >>> honors the provided timeout will lead to a more predictable and >>> user-friendly API. >>> >>> Thanks again for your questions. >>> Let me know if you have any further comments. >>> >>> Best, >>> TengYao >>> >>> Chia-Ping Tsai <chia7...@gmail.com> 於 2024年9月25日 週三 上午4:51寫道： >>> >>>> hi Kirk >>>> >>>>> KT1: Why would a non-Kafka Streams application want to set >>>> leaveGroup=false? Because Kafka Streams manages the group membership >>>> assignment under the covers, it can re-assign partitions to a new >> Consumer >>>> when the old one closes. But in a non-Kafka Streams application, >> doesn’t >>>> this just leave the partitions assigned until the coordinator kicks the >>>> member out of the group? >>>> >>>> That's definitely a great question. I don’t have a clear answer >> regarding >>>> whether leaveGroup=false means not sending a LEAVE_GROUP request, but I >>>> have a crazy idea—what if leaveGroup=false means temporarily leaving >> the >>>> group? My idea is that we could potentially integrate dynamic members >> with >>>> static members. >>>> >>>> Here’s how it could work with changes to the new protocol and consumer: >>>> >>>> 1. The new consumer will exclusively use static members. If a user >>>> doesn't specify one, the new consumer will automatically generate it >>>> (similar to KIP-1082). >>>> 2. Users can define a close option to either leave the group >> permanently >>>> or temporarily. >>>> >>>> This approach offers several advantages: >>>> >>>> 1. It addresses the needs of Kafka Streams (if they start to use new >>>> consumer) >>>> 2. It simplifies the new coordinator, protocol, and consumer, as >> they no >>>> longer need to handle two types of members. >>>> 3. It addresses KIP-1082. The new consumer always sends a >> LEAVE_GROUP >>>> request during closing, even if it hasn't received a response from >> the >>>> server. >>>> >>>>> KT3: Does setting leaveGroup=true carry the *guarantee* that the >> member >>>> will leave the group? I’m currently battling some edge cases where >> close() >>>> is called with a zero timeout and it times out before the consumer can >>>> leave the group cleanly. >>>> >>>> In my opinion, this is a trade-off between 'not honoring the timeout' >> and >>>> 'creating ghost members.' However, I prefer honoring the timeout, as it >>>> provides users with more control over the 'close' process. We should >> trust >>>> that users fully understand the options they choose. >>>> >>>> >>>> Best, >>>> Chia-Ping >>>> >>>> >>>> >>>> Kirk True <k...@kirktrue.pro> 於 2024年9月24日 週二 上午8:37寫道： >>>> >>>>> Hi TengYao, >>>>> >>>>> Thanks for writing up this KIP :) >>>>> >>>>> Questions: >>>>> >>>>> KT1: Why would a non-Kafka Streams application want to set >>>>> leaveGroup=false? Because Kafka Streams manages the group membership >>>>> assignment under the covers, it can re-assign partitions to a new >>>> Consumer >>>>> when the old one closes. But in a non-Kafka Streams application, >> doesn’t >>>>> this just leave the partitions assigned until the coordinator kicks >> the >>>>> member out of the group? >>>>> >>>>> KT2: Can you add the actual API changes to the KIP? >>>>> >>>>> KT3: Does setting leaveGroup=true carry the *guarantee* that the >> member >>>>> will leave the group? I’m currently battling some edge cases where >>>> close() >>>>> is called with a zero timeout and it times out before the consumer >> can >>>>> leave the group cleanly. >>>>> >>>>> Thanks, >>>>> Kirk >>>>> >>>>>> On Sep 22, 2024, at 9:26 PM, TengYao Chi <kiting...@gmail.com> >> wrote: >>>>>> >>>>>> Hello everyone, >>>>>> >>>>>> I would like to start a discussion thread on KIP-1092 >>>>>> <https://cwiki.apache.org/confluence/x/JQstEw>, which proposes >>>> extending >>>>>> Consumer#close with an option to leave the consumer group or not. >>>>>> >>>>>> Please take a look and let me know what you think, and I would >>>> appreciate >>>>>> any suggestions and feedback. >>>>>> >>>>>> Best regards, >>>>>> TengYao >>>>> >>>>> >>>> >>> >>
