Hi Chia-Ping, Is this Connect removal the only one that did not meet the stated requirement?
We do indeed aim not to remove APIs deprecated within the last 12 months before the next major release, but I'm not sure how strictly this is followed (since it's not enforced). If this Connect case is the only one that deviates from that rule, I would recommended reverting it. But if there are many more cases, we may need to look into them in more detail. Ismael On Fri, Feb 28, 2025, 7:10 PM Chia-Ping Tsai <chia7...@apache.org> wrote: > hi Matthias > > thanks for your updates. it looks great and readable. > > Regarding API compatibility, we state "we only provide API backward > compatibility for version 3.7, 3.8, and 3.9" However, within Connect APIs, > we have removed a public API that was deprecated by 3.7 [0][1]. > > Since this KIP will likely include Connect APIs, it's important to discuss > this now. We could: 1) add extra descriptions to highlight the difference > for Connect APIs, or 2) revert the commit that removed the public API for > consistent documentation. > > WDYT? > > [0] > https://github.com/apache/kafka/commit/1c8bb61a43d3ad1fd7a10eb3947342ceba783c4e > [1] > https://github.com/apache/kafka/commit/4f1688742e75a147741fbdc307c5e85d2addb811 > > Best, > Chia-Ping > > On 2025/02/28 22:44:22 "Matthias J. Sax" wrote: > > Thanks for pointing me to test Ismael. Glad to see I just missed it, and > > that we indeed have a test for it. > > > > > > >> Feel free to edit the KIP directly – I'm totally fine with it. Thanks > a lot! > > > > Thanks Kuan-Po! > > > > I did update the wiki page and added more details. I did follow Bruno's > > suggestion, to be a little bit more elaborate, as the KIP page is mainly > > for Kafka maintainers. -- We might want to do it differently in user > > facing docs, but detailed doc updates seems to be out-of-scope for the > > KIP discussion and we can take it on the corresponding PR, to simplify > > it further. > > > > > > I did keep the 2.8.x bridge release for KS for now.; we can still change > > the recommendation and skip 2.8.x as bridge release of course, if we > > think it would be better for users. > > > > > > Curious the get your feedback. > > > > > > -Matthias > > > > > > On 2/28/25 8:15 AM, Kuan Po Tseng wrote: > > > Hi Matthias, > > > > > > Feel free to edit the KIP directly – I'm totally fine with it. Thanks > a lot! > > > > > > Best, > > > Kuan-Po Tseng > > > > > > On 2025/02/28 07:35:10 "Matthias J. Sax" wrote: > > >> Thanks all. I am happy that we are making progress to close this out. > > >> > > >> As mentioned in my (long) email last Friday, I also think that the KIP > > >> makes a lot of implicit assumptions, and it might benefit from begin > > >> more explicit, even explaining things that are "set in stone" already, > > >> and the KIP does not change them (like API deprecation timelines and > > >> guarantees). But as Bruno said, it would help a lot to make it easier > to > > >> understand the KIP. > > >> > > >> Only a very small number of people seems to fully understand all the > > >> nuances, and it took quite a while to collect and curate all the > cases, > > >> to shape this KIP... And we know that a lot of people actually read > > >> KIPs, and the title for KIP-1124 is or sure click-bait... > > >> > > >> I am also happy to edit the wiki page directly if that's ok with Kuan, > > >> to add background information that I believe is missing and important. > > >> > > >> > > >> > > >> My overall understanding of the state of the KIP, ie, its actual > content > > >> seems to be, that there is a high level agreement, and the content of > > >> the KIP is ok. > > >> > > >> The only open question seems to be, if we want to recommend `2.8.x` > as a > > >> bridge release for Kafka Streams or not. As said in my email on > Friday, > > >> I am personally ok either way, with a preference to add `2.8.x` as > > >> bridge release, as it simplifies updating the code by avoiding > > >> compilation errors. With a 2.8.x bridge release, KS application code > > >> would _always_ compile, and the JavaDocs help a lot to guide users how > > >> to migrate off deprecated APIs. -- With a direct upgrade from older > > >> versions to 3.9.x, code does not compile, and there is no guidance as > > >> the old methods (and corresponding JavaDocs) are already removed, > making > > >> it significant harder for user to update their code and make it > compile > > >> again. > > >> > > >> In the end, we would only _recommend_ 2.8.x as a bridge release to > > >> users, and there won't be any claim that an direct upgrade to 3.9.x is > > >> not supported. If one wants to do this, it's tested, and ok. Knock > > >> yourself out -- there is for sure apps out there, that are just a > simple > > >> `builder.stream(...).filter(...).to(...)` the there won't be an API > > >> incompatibility issues. > > >> > > >> But I don't think it would be wise to _recommend_ a direct upgrade to > > >> 3.9.x for the reasons stated, as many apps are more complex. > > >> > > >> Thoughts? > > >> > > >> > > >> > > >> About testing: I did look into the system tests, and it seems we > > >> actually don't really have a real upgrade test for clients, ie, a test > > >> that is running an consumer or producer with some older versions, and > > >> executes a rolling bounce to upgrade the app... > > >> > > >> But maybe such a test is not necessary for clients (compare to Kafka > > >> Streams), and the existing client-broker forward/backward > compatibility > > >> test are good enough? (At least in the last 10 years, it seems this > test > > >> coverage was sufficient?) > > >> > > >> I want to point out though that I agree with Bruno: If we assume that > > >> the client-broker compatibility tests are sufficient to verify client > > >> upgrades, we do effectively/implicitly test upgrading 2.0 (and older > > >> clients) to 4.0 using 3.9 and older brokers. These test coverage does > > >> not exist in a single branch so, but test from different branches > > >> combined, implicitly cover this. Nevertheless, is it tested. > > >> > > >> So it seems both Bruno and Ismael are right, depending how you look at > > >> the problem. In any case, even if we test such an upgrade implicitly, > I > > >> agree that it's good to exclude these upgrades officially as > unsupported. > > >> > > >> > > >> > > >> -Matthias > > >> > > >> > > >> > > >> > > >> On 2/27/25 6:10 AM, Ismael Juma wrote: > > >>> Hi Bruno, > > >>> > > >>> A comment below. > > >>> > > >>> On Thu, Feb 27, 2025, 1:20 AM Bruno Cadonna <cado...@apache.org> > wrote: > > >>> > > >>>> BC2: > > >>>> Another aspect that confuses me in the KIP is the bridge version for > > >>>> clients. If my broker version is 3.9.x, I can do a direct upgrade > from > > >>>> 1.0.x to 4.0.x (let apart the Java API compatibility mentioned > above). > > >>>> The 4.0.x release tests 4.0.x clients against 3.9.x brokers and the > > >>>> 3.9.x release tests 1.0.x clients against 3.9.x brokers. It is not > clear > > >>>> to me why we need a bridge release in this case. > > >>>> > > >>> > > >>> We don't test client upgrades from 1.0.x to 4.0.x. They may work, > but it's > > >>> untested. Again, we are focusing on simplicity versus flexibility > for edge > > >>> cases. The number of people upgrading from 1.0 to 4.0 is very low > (if not > > >>> zero). > > >>> > > >>> That's why I still think the simple description I provided in my > other > > >>> message is the way to go > > >>> > > >>> Deprecated APIs may or may not affect users, so that has to be > evaluated on > > >>> a case by case basis. > > >>> > > >>> Ismael > > >>> > > >> > > >> > > > > >
