Hi Matthias, If phase 2 does indeed warn for use of `group.protocol` config, then removal in phase 3 works for me. I take your point about unnecessary configs, provided we manage the removal in a humane way and don't surprise users.
Thanks, Andrew On 2026/01/28 16:53:06 "Matthias J. Sax" wrote: > About removing the config `group.protocol` with 6.0. I would advocate to > do the removal, as originally proposed. > > If the config serves not purpose any longer (ie, has only one value), > there seems to be no reason to keep it (this also help to reduce the > number of config we have; which is a lot a not easy to navigate for users). > > Also, with AK 5.0, we plan to start to log a WARN when the config is > used (for both values "classic" and "consumer"), and because "consumer" > will be the default, we would expect users to not specifiy the config at > all (also not with value "consumer" any longer), but to rely on the > default. Thus, removing the config with 6.0 would not add any noise for > all "correctly" setup consumers. > > In the end, to avoid WARN logs, with AK 4.3 user would switch from > "classic" to "consumer", and with AK 5.0 remove the config entirely. At > least, this is the desired migration path from my understanding. > > > -Matthias > > On 1/28/26 4:49 AM, Lianet Magrans wrote: > > Thanks all for the great discussion! > > > > Since all the questions were around the same points I will address them > > together: > > > > - I added a section specific to Connect, clarifying that we do want to > > upgrade connect usages of the KafkaConsumer, while the Connect-specific > > implementation of the classic remains untouched for now and unaffected by > > this KIP (to be addressed separately). > > - about removing/keeping the group.protocol config after 6.0: agree that it > > makes sense to not introduce noise to users that were already defining it > > in the right way, so we could keep it and just validate it can only support > > the consumer protocol. Updated. > > - about the timeline and delaying the start of the deprecation cycle, added > > a note on the alternatives with the tradeoffs. One of the main goals is to > > encourage adoption in the KafkaConsumer applications, that's why the > > proposal of starting the deprecation sooner rather than later (not waiting > > on adoption to deprecate). I agree with Matthias' point that as long as we > > allow for a safe deprecation cycle (from 4.3 to 5.0 and extended to 6.0), > > it shouldn't be too aggressive. Thoughts? > > > > Please take a look and let me know your thoughts. Thanks! > > > > Lianet > > > > On Wed, Jan 28, 2026 at 4:52 AM Andrew Schofield <[email protected]> > > wrote: > > > >> HI Lianet et al, > >> I agree with the proposed schedule and starting people on the long path > >> to the new protocol seems like the right decision to me. > >> > >> One comment. > >> > >> AS1: In phase 3, you are going to remove the `group.protocol` config. That > >> seems unnecessary to me. If the user has specified > >> `group.protocol=consumer` > >> as part of their migration onto the new protocol, they are already doing > >> the > >> right thing when we get to AK 6.0 and there’s no reason to remove the > >> config too. I suggest leaving the config but only permitting the value > >> “consumer”. > >> > >> Thanks, > >> Andrew > >> > >>> On 28 Jan 2026, at 03:42, Matthias J. Sax <[email protected]> wrote: > >>> > >>> Thanks for the confirmation on Connect. It does match my understanding. > >> Maybe I did not express myself clearly enough in my last email. -- I agree > >> that it would be good to explain explicitly on the KIP. > >>> > >>> > >>> About "encourage users to use the new `consumer` protocol" vs > >> "deprecating the `classic` protocol", and don't see much of a difference? > >> Also, how do we "encourage" them? Based on experience, a deprecation > >> warning is the best way to encourage them from my POV. > >>> > >>> Also, if we want to make "consumer" default with AK 5.0, it seems > >> reasonable to start the deprecation cycle now. In general, we aim to have a > >> one year deprecation period, so if we deprecate only in one year, eg 4.6, > >> we could only change the default if there is also 4.7 and 4.8 release > >> before 5.0 (or violate the one year guarantee we usually provide). This > >> sounds unnecessary risky. > >>> > >>> If we deprecate with AK 4.3 though, we can make the switch to "consumer" > >> as default with 5.0, even with much fewer 4.x releases, de-risking the > >> process. > >>> > >>> Given that the goal for 5.0 is not to remove "classic", but only to > >> change the default, I don't think it's too aggressive. This KIP is somewhat > >> special as the intended deprecation is not about removal in the next major > >> release, so I think it's fine. > >>> > >>> I would agree that removing "classic" in 5.0 would be too aggressive, > >> but that's not the proposal. > >>> > >>> > >>> > >>> -Matthias > >>> > >>> On 1/27/26 10:23 AM, Ismael Juma wrote: > >>>> Hi David, > >>>> Thanks for the clarification. For the two points you shared: > >>>> 1. I think the key point is that Connect handles this via a separate > >> set of > >>>> classes - not the Consumer. > >>>> 2. Can we make it explicit in the KIP that we will migrate this Connect > >>>> usage to the new rebalance protocol by default in 5.0? > >>>> In addition, I wonder if it's a good idea to deprecate the classic > >> consumer > >>>> in 4.3 - isn't that a bit aggressive? Perhaps deprecation should wait > >> until > >>>> KIP-848 is more widely used. We can still encourage folks to try the > >>>> consumer group protocol. I think a reasonable target for actual > >> deprecation > >>>> would be a 4.x release roughly a year from now. > >>>> Ismael > >>>> On Tue, Jan 27, 2026 at 7:56 AM David Jacot via dev < > >> [email protected]> > >>>> wrote: > >>>>> Ismael, > >>>>> > >>>>> We need to discuss two cases for Connect: > >>>>> > >>>>> 1) Connect uses the classic rebalance protocol to assign its tasks > >> within a > >>>>> Connect cluster. This will keep working until we figure out a plan to > >>>>> migrate this part to another protocol (e.g. Connect protocol following > >>>>> Streams protocol). This is completely orthogonal from the Consumer. > >>>>> > >>>>> 2) Connect uses Consumers in sources/sinks to read from topics. Those > >>>>> consumers can use the consumer rebalance protocol without any issues. > >> They > >>>>> can just follow the upgrade path and use the new protocol when the > >> config > >>>>> changes. > >>>>> > >>>>> This means that in 6.0, Connect's Consumer will use the new rebalance > >>>>> protocol whereas Connect's runtime will continue to use the classic > >>>>> protocol to distribute its tasks. > >>>>> > >>>>> Lianet, we should perhaps clarify this in the KIP. > >>>>> > >>>>> Best, > >>>>> David > >>>>> > >>>>> > >>>>> > >>>>> On Tue, Jan 27, 2026 at 4:27 PM Ismael Juma <[email protected]> wrote: > >>>>> > >>>>>> Lianet, > >>>>>> > >>>>>> Thanks for the KIP. I think this is a desirable goal, but it seems > >>>>>> premature without a plan for Connect. Removing the config for regular > >>>>> users > >>>>>> while allowing it for Connect seems problematic. While we figure the > >>>>>> details of that and the deprecation story for the classic rebalance > >>>>>> protocol, we could still agree to a log line during start-up > >> recommending > >>>>>> `classic` users to switch to the switch to `consumer` (after > >> appropriate > >>>>>> testing). > >>>>>> > >>>>>> Ismael > >>>>>> > >>>>>> On Thu, Jan 22, 2026 at 2:05 PM Lianet Magrans <[email protected]> > >>>>> wrote: > >>>>>> > >>>>>>> Hello, > >>>>>>> > >>>>>>> I would like to start the discussion for KIP-1274. This KIP proposes > >> a > >>>>>>> phased plan to drive a smooth evolution of Java client consumer > >>>>>>> applications towards the next generation of the rebalance protocol > >>>>>>> > >>>>>>> Here is the KIP: > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1274%3A+Deprecate+and+remove+support+for+Classic+rebalance+protocol+in+KafkaConsumer > >>>>>>> > >>>>>>> Looking forward to hearing from you, > >>>>>>> > >>>>>>> Thanks! > >>>>>>> Lianet > >>>>>>> > >>>>>> > >>>>> > >>> > >> > >> > > > >
