Thanks for the confirmation on Connect. It does match my understanding.
Maybe I did not express myself clearly enough in my last email. -- I
agree that it would be good to explain explicitly on the KIP.
About "encourage users to use the new `consumer` protocol" vs
"deprecating the `classic` protocol", and don't see much of a
difference? Also, how do we "encourage" them? Based on experience, a
deprecation warning is the best way to encourage them from my POV.
Also, if we want to make "consumer" default with AK 5.0, it seems
reasonable to start the deprecation cycle now. In general, we aim to
have a one year deprecation period, so if we deprecate only in one year,
eg 4.6, we could only change the default if there is also 4.7 and 4.8
release before 5.0 (or violate the one year guarantee we usually
provide). This sounds unnecessary risky.
If we deprecate with AK 4.3 though, we can make the switch to "consumer"
as default with 5.0, even with much fewer 4.x releases, de-risking the
process.
Given that the goal for 5.0 is not to remove "classic", but only to
change the default, I don't think it's too aggressive. This KIP is
somewhat special as the intended deprecation is not about removal in the
next major release, so I think it's fine.
I would agree that removing "classic" in 5.0 would be too aggressive,
but that's not the proposal.
-Matthias
On 1/27/26 10:23 AM, Ismael Juma wrote:
Hi David,
Thanks for the clarification. For the two points you shared:
1. I think the key point is that Connect handles this via a separate set of
classes - not the Consumer.
2. Can we make it explicit in the KIP that we will migrate this Connect
usage to the new rebalance protocol by default in 5.0?
In addition, I wonder if it's a good idea to deprecate the classic consumer
in 4.3 - isn't that a bit aggressive? Perhaps deprecation should wait until
KIP-848 is more widely used. We can still encourage folks to try the
consumer group protocol. I think a reasonable target for actual deprecation
would be a 4.x release roughly a year from now.
Ismael
On Tue, Jan 27, 2026 at 7:56 AM David Jacot via dev <[email protected]>
wrote:
Ismael,
We need to discuss two cases for Connect:
1) Connect uses the classic rebalance protocol to assign its tasks within a
Connect cluster. This will keep working until we figure out a plan to
migrate this part to another protocol (e.g. Connect protocol following
Streams protocol). This is completely orthogonal from the Consumer.
2) Connect uses Consumers in sources/sinks to read from topics. Those
consumers can use the consumer rebalance protocol without any issues. They
can just follow the upgrade path and use the new protocol when the config
changes.
This means that in 6.0, Connect's Consumer will use the new rebalance
protocol whereas Connect's runtime will continue to use the classic
protocol to distribute its tasks.
Lianet, we should perhaps clarify this in the KIP.
Best,
David
On Tue, Jan 27, 2026 at 4:27 PM Ismael Juma <[email protected]> wrote:
Lianet,
Thanks for the KIP. I think this is a desirable goal, but it seems
premature without a plan for Connect. Removing the config for regular
users
while allowing it for Connect seems problematic. While we figure the
details of that and the deprecation story for the classic rebalance
protocol, we could still agree to a log line during start-up recommending
`classic` users to switch to the switch to `consumer` (after appropriate
testing).
Ismael
On Thu, Jan 22, 2026 at 2:05 PM Lianet Magrans <[email protected]>
wrote:
Hello,
I would like to start the discussion for KIP-1274. This KIP proposes a
phased plan to drive a smooth evolution of Java client consumer
applications towards the next generation of the rebalance protocol
Here is the KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1274%3A+Deprecate+and+remove+support+for+Classic+rebalance+protocol+in+KafkaConsumer
Looking forward to hearing from you,
Thanks!
Lianet
