Thanks for your attention and reply. Having chatted with Guozhang Wang at KAFKA-12478 before, I came up with an idea similar to yours. It's just not implemented on the client side, but on the server side: Firstly, find out all the groups subscribed to this topic before extending partitions, and then let these groups commit an initial offset 0 for these new expanded partitions (also using adminClient). Finally, the real process of adding partitions is carried out. In this way, the problem can also be completely solved.
Best, hudeqi "Matthew Howlett" <m...@confluent.io.INVALID>写道： > My first reaction also is that the proposed configuration is surely too > complicated. > > It seems like an ideal solution from a usability perspective (always a good > place to start) would be if the consumer just automatically behaved in this > way. To make that work: > 1. auto.offset.reset=latest would need to behave like > auto.offset.reset=earliest in the case where a consumer is in a group, and > is assigned a newly created partition. This might seem a bit too "magic", > but from the perspective of the group, I think it makes conceptual sense > and people wouldn't find it surprising. Also, I don't think anyone would be > relying on the old behavior. > 2. The group would need to detect the newly created partitions and > rebalance pretty quickly (this is not the case currently). The longer the > delay, the more tenuous the idea of changing the auto.offset.reset behavior > in this special circumstance. > > I have a feeling this approach has implementation challenges (haven't > thought deeply), just throwing it out there. > > > On Wed, Jun 29, 2022 at 4:57 AM David Jacot <da...@apache.org> wrote: > > > Thanks for the KIP. > > > > I read it and I am also worried by the complexity of the new > > configurations. They are not easy to grasp. I need to digest it a bit more, > > I think. > > > > Best, > > David > > > > Le mer. 29 juin 2022 à 02:25, Matthias J. Sax <mj...@apache.org> a écrit : > > > > > Thanks for the KIP. > > > > > > I don't think I fully digested the proposal yet, but my first reaction > > > is: this is quite complicated. Frankly, I am worried about complexity > > > and usability. > > > > > > Especially the option `safe_latest` is a "weird" one IMHO, and `nearest` > > > is even more complex. > > > > > > The problem at hand (as I understand it from the Jira) is a real one, > > > but I am wondering if it would be something that should be addressed by > > > the application? If you pass in strategy `none`, and a new partition is > > > added, you can react to it by custom code. For regular startup you can > > > still go with "latest" to avoid reprocessing the history. > > > > > > Adding "latest/earliest_on_start" seems useful, as it seems to also > > > address https://issues.apache.org/jira/browse/KAFKA-3370 > > > > > > > > > -Matthias > > > > > > > > > On 6/7/22 12:55 AM, hudeqi wrote: > > > > I think so too, what about Guozhang Wang and Luke Chen? Can I initiate > > a > > > voting process? > > > > > > > > Best, > > > > hudeqi > > > > > > > > > -----原始邮件----- > > > > > 发件人: "邓子明" <dengzim...@growingio.com> > > > > > 发送时间: 2022-06-07 10:23:37 (星期二) > > > > > 收件人: dev@kafka.apache.org > > > > > 抄送: > > > > > 主题: Re: [DISCUSS] KIP-842: Add richer group offset reset > > mechanisms > > > > > > > > > </dengzim...@growingio.com> > > > > >
