> For this case, what was the reason that the 3-prior-release rule was ignored?
According to KIP-970 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-970%3A+Deprecate+and+remove+Connect%27s+redundant+task+configurations+endpoint), it seems the main reason to avoid "a bit of a maintenance burden to have two different endpoints for the same purpose". Best, Chia-Ping Matthias J. Sax <mj...@apache.org> 於 2025年3月3日 週一 上午3:57寫道： > >> deliberately. > > For this case, what was the reason that the 3-prior-release rule was > ignored? > > > -Matthias > > On 3/1/25 11:29 PM, Chia-Ping Tsai wrote: > > hi > > > >> If this Connect case is the only one that deviates from that rule, I > would > > recommended reverting it. But if there are many more cases, we may need > to > > look into them in more detail. > > > > I briefly checked the commit history, and this is the only one that needs > > to be highlighted. Fortunately, it is easy to revert. > > > >> why was it removed? Oversight (ie, accidentally > > too early) or deliberately? > > > > deliberately. > > > > I have left a common on the PR ( > > https://github.com/apache/kafka/pull/17412#issuecomment-2692600494) > > > > Best, > > Chia-Ping > > > > > > Matthias J. Sax <mj...@apache.org> 於 2025年3月2日 週日 上午6:32寫道： > > > >> Don't have a strong opinion about the Connect case. > >> > >> I guess the question is: why was it removed? Oversight (ie, accidentally > >> too early) or deliberately? Also, what is the impact if we keep the API > >> and revert the removal? > >> > >> IMHO, the 3-releases-rule can have exceptions, if there are good reason > >> for it. I just don't have enough context about the change to judge. > >> > >> > >> > >>>> One more thing, 3.7 was released in Feb 2024 and hence it will be more > >> than > >>>> 12 months by the time 4.0 is released. Technically, it would be ok to > >>>> remove APIs deprecated in 3.7. > >> > >> Not sure if this is a strong argument. We cannot time releases totally > >> accurately, so the 3-prior-releases rule should be the "source of truth" > >> IMHO, that just practically translated to "about 1 year". But as I said > >> above, I also don't think we need to be dogmatic about it. > >> > >> > >> > >> -Matthias > >> > >> On 3/1/25 8:37 AM, Ismael Juma wrote: > >>> One more thing, 3.7 was released in Feb 2024 and hence it will be more > >> than > >>> 12 months by the time 4.0 is released. Technically, it would be ok to > >>> remove APIs deprecated in 3.7. > >>> > >>> Ismael > >>> > >>> On Sat, Mar 1, 2025, 8:34 AM Ismael Juma <m...@ismaeljuma.com> wrote: > >>> > >>>> Hi Chia-Ping, > >>>> > >>>> Is this Connect removal the only one that did not meet the stated > >>>> requirement? > >>>> > >>>> We do indeed aim not to remove APIs deprecated within the last 12 > months > >>>> before the next major release, but I'm not sure how strictly this is > >>>> followed (since it's not enforced). > >>>> > >>>> If this Connect case is the only one that deviates from that rule, I > >> would > >>>> recommended reverting it. But if there are many more cases, we may > need > >> to > >>>> look into them in more detail. > >>>> > >>>> Ismael > >>>> > >>>> On Fri, Feb 28, 2025, 7:10 PM Chia-Ping Tsai <chia7...@apache.org> > >> wrote: > >>>> > >>>>> hi Matthias > >>>>> > >>>>> thanks for your updates. it looks great and readable. > >>>>> > >>>>> Regarding API compatibility, we state "we only provide API backward > >>>>> compatibility for version 3.7, 3.8, and 3.9" However, within Connect > >> APIs, > >>>>> we have removed a public API that was deprecated by 3.7 [0][1]. > >>>>> > >>>>> Since this KIP will likely include Connect APIs, it's important to > >>>>> discuss this now. We could: 1) add extra descriptions to highlight > the > >>>>> difference for Connect APIs, or 2) revert the commit that removed the > >>>>> public API for consistent documentation. > >>>>> > >>>>> WDYT? > >>>>> > >>>>> [0] > >>>>> > >> > https://github.com/apache/kafka/commit/1c8bb61a43d3ad1fd7a10eb3947342ceba783c4e > >>>>> [1] > >>>>> > >> > https://github.com/apache/kafka/commit/4f1688742e75a147741fbdc307c5e85d2addb811 > >>>>> > >>>>> Best, > >>>>> Chia-Ping > >>>>> > >>>>> On 2025/02/28 22:44:22 "Matthias J. Sax" wrote: > >>>>>> Thanks for pointing me to test Ismael. Glad to see I just missed it, > >>>>> and > >>>>>> that we indeed have a test for it. > >>>>>> > >>>>>> > >>>>>>>> Feel free to edit the KIP directly – I'm totally fine with it. > >>>>> Thanks a lot! > >>>>>> > >>>>>> Thanks Kuan-Po! > >>>>>> > >>>>>> I did update the wiki page and added more details. I did follow > >> Bruno's > >>>>>> suggestion, to be a little bit more elaborate, as the KIP page is > >>>>> mainly > >>>>>> for Kafka maintainers. -- We might want to do it differently in user > >>>>>> facing docs, but detailed doc updates seems to be out-of-scope for > the > >>>>>> KIP discussion and we can take it on the corresponding PR, to > simplify > >>>>>> it further. > >>>>>> > >>>>>> > >>>>>> I did keep the 2.8.x bridge release for KS for now.; we can still > >>>>> change > >>>>>> the recommendation and skip 2.8.x as bridge release of course, if we > >>>>>> think it would be better for users. > >>>>>> > >>>>>> > >>>>>> Curious the get your feedback. > >>>>>> > >>>>>> > >>>>>> -Matthias > >>>>>> > >>>>>> > >>>>>> On 2/28/25 8:15 AM, Kuan Po Tseng wrote: > >>>>>>> Hi Matthias, > >>>>>>> > >>>>>>> Feel free to edit the KIP directly – I'm totally fine with it. > Thanks > >>>>> a lot! > >>>>>>> > >>>>>>> Best, > >>>>>>> Kuan-Po Tseng > >>>>>>> > >>>>>>> On 2025/02/28 07:35:10 "Matthias J. Sax" wrote: > >>>>>>>> Thanks all. I am happy that we are making progress to close this > >> out. > >>>>>>>> > >>>>>>>> As mentioned in my (long) email last Friday, I also think that the > >>>>> KIP > >>>>>>>> makes a lot of implicit assumptions, and it might benefit from > begin > >>>>>>>> more explicit, even explaining things that are "set in stone" > >>>>> already, > >>>>>>>> and the KIP does not change them (like API deprecation timelines > and > >>>>>>>> guarantees). But as Bruno said, it would help a lot to make it > >>>>> easier to > >>>>>>>> understand the KIP. > >>>>>>>> > >>>>>>>> Only a very small number of people seems to fully understand all > the > >>>>>>>> nuances, and it took quite a while to collect and curate all the > >>>>> cases, > >>>>>>>> to shape this KIP... And we know that a lot of people actually > read > >>>>>>>> KIPs, and the title for KIP-1124 is or sure click-bait... > >>>>>>>> > >>>>>>>> I am also happy to edit the wiki page directly if that's ok with > >>>>> Kuan, > >>>>>>>> to add background information that I believe is missing and > >>>>> important. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> My overall understanding of the state of the KIP, ie, its actual > >>>>> content > >>>>>>>> seems to be, that there is a high level agreement, and the content > >> of > >>>>>>>> the KIP is ok. > >>>>>>>> > >>>>>>>> The only open question seems to be, if we want to recommend > `2.8.x` > >>>>> as a > >>>>>>>> bridge release for Kafka Streams or not. As said in my email on > >>>>> Friday, > >>>>>>>> I am personally ok either way, with a preference to add `2.8.x` as > >>>>>>>> bridge release, as it simplifies updating the code by avoiding > >>>>>>>> compilation errors. With a 2.8.x bridge release, KS application > code > >>>>>>>> would _always_ compile, and the JavaDocs help a lot to guide users > >>>>> how > >>>>>>>> to migrate off deprecated APIs. -- With a direct upgrade from > older > >>>>>>>> versions to 3.9.x, code does not compile, and there is no guidance > >> as > >>>>>>>> the old methods (and corresponding JavaDocs) are already removed, > >>>>> making > >>>>>>>> it significant harder for user to update their code and make it > >>>>> compile > >>>>>>>> again. > >>>>>>>> > >>>>>>>> In the end, we would only _recommend_ 2.8.x as a bridge release to > >>>>>>>> users, and there won't be any claim that an direct upgrade to > 3.9.x > >>>>> is > >>>>>>>> not supported. If one wants to do this, it's tested, and ok. Knock > >>>>>>>> yourself out -- there is for sure apps out there, that are just a > >>>>> simple > >>>>>>>> `builder.stream(...).filter(...).to(...)` the there won't be an > API > >>>>>>>> incompatibility issues. > >>>>>>>> > >>>>>>>> But I don't think it would be wise to _recommend_ a direct upgrade > >> to > >>>>>>>> 3.9.x for the reasons stated, as many apps are more complex. > >>>>>>>> > >>>>>>>> Thoughts? > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> About testing: I did look into the system tests, and it seems we > >>>>>>>> actually don't really have a real upgrade test for clients, ie, a > >>>>> test > >>>>>>>> that is running an consumer or producer with some older versions, > >> and > >>>>>>>> executes a rolling bounce to upgrade the app... > >>>>>>>> > >>>>>>>> But maybe such a test is not necessary for clients (compare to > Kafka > >>>>>>>> Streams), and the existing client-broker forward/backward > >>>>> compatibility > >>>>>>>> test are good enough? (At least in the last 10 years, it seems > this > >>>>> test > >>>>>>>> coverage was sufficient?) > >>>>>>>> > >>>>>>>> I want to point out though that I agree with Bruno: If we assume > >> that > >>>>>>>> the client-broker compatibility tests are sufficient to verify > >> client > >>>>>>>> upgrades, we do effectively/implicitly test upgrading 2.0 (and > older > >>>>>>>> clients) to 4.0 using 3.9 and older brokers. These test coverage > >> does > >>>>>>>> not exist in a single branch so, but test from different branches > >>>>>>>> combined, implicitly cover this. Nevertheless, is it tested. > >>>>>>>> > >>>>>>>> So it seems both Bruno and Ismael are right, depending how you > look > >>>>> at > >>>>>>>> the problem. In any case, even if we test such an upgrade > >>>>> implicitly, I > >>>>>>>> agree that it's good to exclude these upgrades officially as > >>>>> unsupported. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -Matthias > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On 2/27/25 6:10 AM, Ismael Juma wrote: > >>>>>>>>> Hi Bruno, > >>>>>>>>> > >>>>>>>>> A comment below. > >>>>>>>>> > >>>>>>>>> On Thu, Feb 27, 2025, 1:20 AM Bruno Cadonna <cado...@apache.org> > >>>>> wrote: > >>>>>>>>> > >>>>>>>>>> BC2: > >>>>>>>>>> Another aspect that confuses me in the KIP is the bridge version > >>>>> for > >>>>>>>>>> clients. If my broker version is 3.9.x, I can do a direct > upgrade > >>>>> from > >>>>>>>>>> 1.0.x to 4.0.x (let apart the Java API compatibility mentioned > >>>>> above). > >>>>>>>>>> The 4.0.x release tests 4.0.x clients against 3.9.x brokers and > >> the > >>>>>>>>>> 3.9.x release tests 1.0.x clients against 3.9.x brokers. It is > not > >>>>> clear > >>>>>>>>>> to me why we need a bridge release in this case. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> We don't test client upgrades from 1.0.x to 4.0.x. They may work, > >>>>> but it's > >>>>>>>>> untested. Again, we are focusing on simplicity versus flexibility > >>>>> for edge > >>>>>>>>> cases. The number of people upgrading from 1.0 to 4.0 is very low > >>>>> (if not > >>>>>>>>> zero). > >>>>>>>>> > >>>>>>>>> That's why I still think the simple description I provided in my > >>>>> other > >>>>>>>>> message is the way to go > >>>>>>>>> > >>>>>>>>> Deprecated APIs may or may not affect users, so that has to be > >>>>> evaluated on > >>>>>>>>> a case by case basis. > >>>>>>>>> > >>>>>>>>> Ismael > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > >> > > > >
