Hi Matthias, Two quick comments:
1. Yes, please help edit the KIP directly. It's easy enough to revert if Kuan disagrees. 2. Regarding client upgrades, the test in https://github.com/apache/kafka/pull/18193/files is an example of test that only starts from 2.1 (since we did not merge #18193). Ismael On Thu, Feb 27, 2025 at 11:36 PM Matthias J. Sax <mj...@apache.org> wrote: > Thanks all. I am happy that we are making progress to close this out. > > As mentioned in my (long) email last Friday, I also think that the KIP > makes a lot of implicit assumptions, and it might benefit from begin > more explicit, even explaining things that are "set in stone" already, > and the KIP does not change them (like API deprecation timelines and > guarantees). But as Bruno said, it would help a lot to make it easier to > understand the KIP. > > Only a very small number of people seems to fully understand all the > nuances, and it took quite a while to collect and curate all the cases, > to shape this KIP... And we know that a lot of people actually read > KIPs, and the title for KIP-1124 is or sure click-bait... > > I am also happy to edit the wiki page directly if that's ok with Kuan, > to add background information that I believe is missing and important. > > > > My overall understanding of the state of the KIP, ie, its actual content > seems to be, that there is a high level agreement, and the content of > the KIP is ok. > > The only open question seems to be, if we want to recommend `2.8.x` as a > bridge release for Kafka Streams or not. As said in my email on Friday, > I am personally ok either way, with a preference to add `2.8.x` as > bridge release, as it simplifies updating the code by avoiding > compilation errors. With a 2.8.x bridge release, KS application code > would _always_ compile, and the JavaDocs help a lot to guide users how > to migrate off deprecated APIs. -- With a direct upgrade from older > versions to 3.9.x, code does not compile, and there is no guidance as > the old methods (and corresponding JavaDocs) are already removed, making > it significant harder for user to update their code and make it compile > again. > > In the end, we would only _recommend_ 2.8.x as a bridge release to > users, and there won't be any claim that an direct upgrade to 3.9.x is > not supported. If one wants to do this, it's tested, and ok. Knock > yourself out -- there is for sure apps out there, that are just a simple > `builder.stream(...).filter(...).to(...)` the there won't be an API > incompatibility issues. > > But I don't think it would be wise to _recommend_ a direct upgrade to > 3.9.x for the reasons stated, as many apps are more complex. > > Thoughts? > > > > About testing: I did look into the system tests, and it seems we > actually don't really have a real upgrade test for clients, ie, a test > that is running an consumer or producer with some older versions, and > executes a rolling bounce to upgrade the app... > > But maybe such a test is not necessary for clients (compare to Kafka > Streams), and the existing client-broker forward/backward compatibility > test are good enough? (At least in the last 10 years, it seems this test > coverage was sufficient?) > > I want to point out though that I agree with Bruno: If we assume that > the client-broker compatibility tests are sufficient to verify client > upgrades, we do effectively/implicitly test upgrading 2.0 (and older > clients) to 4.0 using 3.9 and older brokers. These test coverage does > not exist in a single branch so, but test from different branches > combined, implicitly cover this. Nevertheless, is it tested. > > So it seems both Bruno and Ismael are right, depending how you look at > the problem. In any case, even if we test such an upgrade implicitly, I > agree that it's good to exclude these upgrades officially as unsupported. > > > > -Matthias > > > > > On 2/27/25 6:10 AM, Ismael Juma wrote: > > Hi Bruno, > > > > A comment below. > > > > On Thu, Feb 27, 2025, 1:20 AM Bruno Cadonna <cado...@apache.org> wrote: > > > >> BC2: > >> Another aspect that confuses me in the KIP is the bridge version for > >> clients. If my broker version is 3.9.x, I can do a direct upgrade from > >> 1.0.x to 4.0.x (let apart the Java API compatibility mentioned above). > >> The 4.0.x release tests 4.0.x clients against 3.9.x brokers and the > >> 3.9.x release tests 1.0.x clients against 3.9.x brokers. It is not clear > >> to me why we need a bridge release in this case. > >> > > > > We don't test client upgrades from 1.0.x to 4.0.x. They may work, but > it's > > untested. Again, we are focusing on simplicity versus flexibility for > edge > > cases. The number of people upgrading from 1.0 to 4.0 is very low (if not > > zero). > > > > That's why I still think the simple description I provided in my other > > message is the way to go > > > > Deprecated APIs may or may not affect users, so that has to be evaluated > on > > a case by case basis. > > > > Ismael > > > >
