Thanks all. I am happy that we are making progress to close this out.
As mentioned in my (long) email last Friday, I also think that the KIP
makes a lot of implicit assumptions, and it might benefit from begin
more explicit, even explaining things that are "set in stone" already,
and the KIP does not change them (like API deprecation timelines and
guarantees). But as Bruno said, it would help a lot to make it easier to
understand the KIP.
Only a very small number of people seems to fully understand all the
nuances, and it took quite a while to collect and curate all the cases,
to shape this KIP... And we know that a lot of people actually read
KIPs, and the title for KIP-1124 is or sure click-bait...
I am also happy to edit the wiki page directly if that's ok with Kuan,
to add background information that I believe is missing and important.
My overall understanding of the state of the KIP, ie, its actual content
seems to be, that there is a high level agreement, and the content of
the KIP is ok.
The only open question seems to be, if we want to recommend `2.8.x` as a
bridge release for Kafka Streams or not. As said in my email on Friday,
I am personally ok either way, with a preference to add `2.8.x` as
bridge release, as it simplifies updating the code by avoiding
compilation errors. With a 2.8.x bridge release, KS application code
would _always_ compile, and the JavaDocs help a lot to guide users how
to migrate off deprecated APIs. -- With a direct upgrade from older
versions to 3.9.x, code does not compile, and there is no guidance as
the old methods (and corresponding JavaDocs) are already removed, making
it significant harder for user to update their code and make it compile
again.
In the end, we would only _recommend_ 2.8.x as a bridge release to
users, and there won't be any claim that an direct upgrade to 3.9.x is
not supported. If one wants to do this, it's tested, and ok. Knock
yourself out -- there is for sure apps out there, that are just a simple
`builder.stream(...).filter(...).to(...)` the there won't be an API
incompatibility issues.
But I don't think it would be wise to _recommend_ a direct upgrade to
3.9.x for the reasons stated, as many apps are more complex.
Thoughts?
About testing: I did look into the system tests, and it seems we
actually don't really have a real upgrade test for clients, ie, a test
that is running an consumer or producer with some older versions, and
executes a rolling bounce to upgrade the app...
But maybe such a test is not necessary for clients (compare to Kafka
Streams), and the existing client-broker forward/backward compatibility
test are good enough? (At least in the last 10 years, it seems this test
coverage was sufficient?)
I want to point out though that I agree with Bruno: If we assume that
the client-broker compatibility tests are sufficient to verify client
upgrades, we do effectively/implicitly test upgrading 2.0 (and older
clients) to 4.0 using 3.9 and older brokers. These test coverage does
not exist in a single branch so, but test from different branches
combined, implicitly cover this. Nevertheless, is it tested.
So it seems both Bruno and Ismael are right, depending how you look at
the problem. In any case, even if we test such an upgrade implicitly, I
agree that it's good to exclude these upgrades officially as unsupported.
-Matthias
On 2/27/25 6:10 AM, Ismael Juma wrote:
Hi Bruno,
A comment below.
On Thu, Feb 27, 2025, 1:20 AM Bruno Cadonna <cado...@apache.org> wrote:
BC2:
Another aspect that confuses me in the KIP is the bridge version for
clients. If my broker version is 3.9.x, I can do a direct upgrade from
1.0.x to 4.0.x (let apart the Java API compatibility mentioned above).
The 4.0.x release tests 4.0.x clients against 3.9.x brokers and the
3.9.x release tests 1.0.x clients against 3.9.x brokers. It is not clear
to me why we need a bridge release in this case.
We don't test client upgrades from 1.0.x to 4.0.x. They may work, but it's
untested. Again, we are focusing on simplicity versus flexibility for edge
cases. The number of people upgrading from 1.0 to 4.0 is very low (if not
zero).
That's why I still think the simple description I provided in my other
message is the way to go
Deprecated APIs may or may not affect users, so that has to be evaluated on
a case by case basis.
Ismael
