> And if I understood correctly, this is already done for COOPERATIVE anyway.
Right. We'll keep the partitions status for prev-owned partitions in Cooperative protocol. Thanks. Luke On Thu, Feb 17, 2022 at 2:20 PM Ryan Leslie (BLOOMBERG/ 919 3RD A) < rles...@bloomberg.net> wrote: > Thanks Luke. I’ve always passed a duration of 0 if polling while paused. > The logic being that the poll is only used to preserve group membership and > there’s no need to wait for anything. That probably explains why I would > have been unlikely to ever see this behavior. > > I agree with changing the implementation to guarantee that a pause is > preserved after rebalance. That is in line with how the API has always been > documented, despite the comments in the original JIRA. And if I understood > correctly, this is already done for COOPERATIVE anyway. > > ----- Original Message ----- > From: Luke Chen <dev@kafka.apache.org> > To: RYAN LESLIE, dev@kafka.apache.org > At: 02/16/22 22:02:41 UTC-05:00 > > > Hi Ryan, > > > I've used pause() between version 0.10 and 2.7 with EAGER protocol only > and have not yet observed cases where poll() returned messages unless > resume() had been called. > > Yes, this issue only happens when group rebalance is triggered, the pause > flag will be cleared. And if the Consumer#poll() duration is long enough to > fetch records after rebalance completes, the records would return. > > @Riven, is my understanding correct? > And also, as discussed with Guozhang, you can close this KIP discussion and > start to implement it directly. > > Thank you. > Luke > > On Tue, Feb 15, 2022 at 7:46 AM Ryan Leslie (BLOOMBERG/ 919 3RD A) < > rles...@bloomberg.net> wrote: > > > Apologies, but as worded KIP-822 / KAFKA-13425 are a bit challenging to > > follow. I've used pause() between version 0.10 and 2.7 with EAGER > protocol > > only and have not yet observed cases where poll() returned messages > unless > > resume() had been called. The KIP seems to contradict this though, and > > indicate that poll() does return messages for EAGER protocol. Maybe a > > summary of the conditions where this bug occurs could help? > > > > From: dev@kafka.apache.org At: 02/14/22 17:32:56 UTC-5:00To: > > dev@kafka.apache.org > > Subject: Re: [DISCUSS] KIP-822: Optimize the semantics of > > KafkaConsumer#pause to be consistent between the two RebalanceProtocols > > > > Hi Luke, > > > > Yeah my understanding of the KIP are from > > https://issues.apache.org/jira/browse/KAFKA-13463 and > > https://issues.apache.org/jira/browse/KAFKA-13425, and I think we and > > Riven are on the same page. > > > > If the changes are just within the internal classes, I think we can just > > treat it as a normal bug fix and not need a KIP for it. > > > > > > Guozhang > > > > On Mon, Feb 14, 2022 at 2:22 AM Luke Chen <show...@gmail.com> wrote: > > > > > Hi Guozhang, > > > > > > In short, what Riven wants to do, is that we keep the `pause` flag > after > > > rebalanced, while before this change, Rebalance does not preserve > > > pause/resume state as described in KAFKA-2350 > > > <https://issues.apache.org/jira/browse/KAFKA-2350>. > > > In this case, do you think we can skip the KIP and move on the > > > implementation? > > > > > > I'm good if we need a KIP or not, just want to make it clear in case it > > > breaks any existing expectation/behavior. > > > > > > Thank you. > > > Luke > > > > > > On Mon, Feb 14, 2022 at 10:09 AM Guozhang Wang <wangg...@gmail.com> > > wrote: > > > > > > > Hello Riven, > > > > > > > > > > > > Thanks for bringing this proposal. As we discussed on the JIRA I'm > > > > personally in favor of this fix. But if all the proposed changes are > in > > > > `ConsumerCoordinator`, then we do not need a KIP since that class is > > > > internal only. > > > > > > > > > > > > Guozhang > > > > > > > > On Sat, Feb 12, 2022 at 1:35 AM Riven Sun <riven....@zoom.us.invalid > > > > > > wrote: > > > > > > > > > Sorry, I sent this email via GMail. Refer to the contents of other > > > > people's > > > > > DISSCUSS emails. Mistakenly introduced someone else's KIP. > > > > > > > > > > The KIP related to this DISCUSS is > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199534763 > > > > > > > > > > Thank you for your kindness > > > > > RivenSun > > > > > > > > > > On Sat, Feb 12, 2022 at 5:32 PM Riven Sun <riven....@zoom.us> > wrote: > > > > > > > > > > > > > > > > >> Sorry, I sent this email via GMail. Refer to the contents of > other > > > > > >> people's DISSCUSS emails. Mistakenly introduced someone else's > > KIP. > > > > > >> > > > > > >> The KIP related to this DISCUSS is > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199534763 > > > > > >> > > > > > >> Thank you for your kindness > > > > > >> RivenSun > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > -- > > > > -- Guozhang > > > > > > > > > > > > > -- > > -- Guozhang > > > > > > >
