Thanks all for the great discussion! Since all the questions were around the same points I will address them together:
- I added a section specific to Connect, clarifying that we do want to upgrade connect usages of the KafkaConsumer, while the Connect-specific implementation of the classic remains untouched for now and unaffected by this KIP (to be addressed separately). - about removing/keeping the group.protocol config after 6.0: agree that it makes sense to not introduce noise to users that were already defining it in the right way, so we could keep it and just validate it can only support the consumer protocol. Updated. - about the timeline and delaying the start of the deprecation cycle, added a note on the alternatives with the tradeoffs. One of the main goals is to encourage adoption in the KafkaConsumer applications, that's why the proposal of starting the deprecation sooner rather than later (not waiting on adoption to deprecate). I agree with Matthias' point that as long as we allow for a safe deprecation cycle (from 4.3 to 5.0 and extended to 6.0), it shouldn't be too aggressive. Thoughts? Please take a look and let me know your thoughts. Thanks! Lianet On Wed, Jan 28, 2026 at 4:52 AM Andrew Schofield <[email protected]> wrote: > HI Lianet et al, > I agree with the proposed schedule and starting people on the long path > to the new protocol seems like the right decision to me. > > One comment. > > AS1: In phase 3, you are going to remove the `group.protocol` config. That > seems unnecessary to me. If the user has specified > `group.protocol=consumer` > as part of their migration onto the new protocol, they are already doing > the > right thing when we get to AK 6.0 and there’s no reason to remove the > config too. I suggest leaving the config but only permitting the value > “consumer”. > > Thanks, > Andrew > > > On 28 Jan 2026, at 03:42, Matthias J. Sax <[email protected]> wrote: > > > > Thanks for the confirmation on Connect. It does match my understanding. > Maybe I did not express myself clearly enough in my last email. -- I agree > that it would be good to explain explicitly on the KIP. > > > > > > About "encourage users to use the new `consumer` protocol" vs > "deprecating the `classic` protocol", and don't see much of a difference? > Also, how do we "encourage" them? Based on experience, a deprecation > warning is the best way to encourage them from my POV. > > > > Also, if we want to make "consumer" default with AK 5.0, it seems > reasonable to start the deprecation cycle now. In general, we aim to have a > one year deprecation period, so if we deprecate only in one year, eg 4.6, > we could only change the default if there is also 4.7 and 4.8 release > before 5.0 (or violate the one year guarantee we usually provide). This > sounds unnecessary risky. > > > > If we deprecate with AK 4.3 though, we can make the switch to "consumer" > as default with 5.0, even with much fewer 4.x releases, de-risking the > process. > > > > Given that the goal for 5.0 is not to remove "classic", but only to > change the default, I don't think it's too aggressive. This KIP is somewhat > special as the intended deprecation is not about removal in the next major > release, so I think it's fine. > > > > I would agree that removing "classic" in 5.0 would be too aggressive, > but that's not the proposal. > > > > > > > > -Matthias > > > > On 1/27/26 10:23 AM, Ismael Juma wrote: > >> Hi David, > >> Thanks for the clarification. For the two points you shared: > >> 1. I think the key point is that Connect handles this via a separate > set of > >> classes - not the Consumer. > >> 2. Can we make it explicit in the KIP that we will migrate this Connect > >> usage to the new rebalance protocol by default in 5.0? > >> In addition, I wonder if it's a good idea to deprecate the classic > consumer > >> in 4.3 - isn't that a bit aggressive? Perhaps deprecation should wait > until > >> KIP-848 is more widely used. We can still encourage folks to try the > >> consumer group protocol. I think a reasonable target for actual > deprecation > >> would be a 4.x release roughly a year from now. > >> Ismael > >> On Tue, Jan 27, 2026 at 7:56 AM David Jacot via dev < > [email protected]> > >> wrote: > >>> Ismael, > >>> > >>> We need to discuss two cases for Connect: > >>> > >>> 1) Connect uses the classic rebalance protocol to assign its tasks > within a > >>> Connect cluster. This will keep working until we figure out a plan to > >>> migrate this part to another protocol (e.g. Connect protocol following > >>> Streams protocol). This is completely orthogonal from the Consumer. > >>> > >>> 2) Connect uses Consumers in sources/sinks to read from topics. Those > >>> consumers can use the consumer rebalance protocol without any issues. > They > >>> can just follow the upgrade path and use the new protocol when the > config > >>> changes. > >>> > >>> This means that in 6.0, Connect's Consumer will use the new rebalance > >>> protocol whereas Connect's runtime will continue to use the classic > >>> protocol to distribute its tasks. > >>> > >>> Lianet, we should perhaps clarify this in the KIP. > >>> > >>> Best, > >>> David > >>> > >>> > >>> > >>> On Tue, Jan 27, 2026 at 4:27 PM Ismael Juma <[email protected]> wrote: > >>> > >>>> Lianet, > >>>> > >>>> Thanks for the KIP. I think this is a desirable goal, but it seems > >>>> premature without a plan for Connect. Removing the config for regular > >>> users > >>>> while allowing it for Connect seems problematic. While we figure the > >>>> details of that and the deprecation story for the classic rebalance > >>>> protocol, we could still agree to a log line during start-up > recommending > >>>> `classic` users to switch to the switch to `consumer` (after > appropriate > >>>> testing). > >>>> > >>>> Ismael > >>>> > >>>> On Thu, Jan 22, 2026 at 2:05 PM Lianet Magrans <[email protected]> > >>> wrote: > >>>> > >>>>> Hello, > >>>>> > >>>>> I would like to start the discussion for KIP-1274. This KIP proposes > a > >>>>> phased plan to drive a smooth evolution of Java client consumer > >>>>> applications towards the next generation of the rebalance protocol > >>>>> > >>>>> Here is the KIP: > >>>>> > >>>>> > >>>> > >>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1274%3A+Deprecate+and+remove+support+for+Classic+rebalance+protocol+in+KafkaConsumer > >>>>> > >>>>> Looking forward to hearing from you, > >>>>> > >>>>> Thanks! > >>>>> Lianet > >>>>> > >>>> > >>> > > > >
