Thanks Chia-Ping for the summary! I'll take some time to refine the KIP as you 
suggested.

I do have one little question about this:

> 1) Client-Broker Protocol Compatibility: Ensure seamless interoperability 
> between 5.x clients and brokers, as well as with clients and brokers in the 
> [4.x, 6.x] range.

When we're talking about two major versions, should it be [4.x, 5.x] or [4.x, 
6.x]? The latter seems to refer to three major versions, doesn't it?"


On 2025/01/19 10:15:11 Chia-Ping Tsai wrote:
> Dear all,
> 
> > only support upgrades between 2 major version, and also 
> limit client-broker compatibility to two major version.
> 
> I wholeheartedly support this concept due to its inherent simplicity. 
> However, a primary concern arises regarding third-party client libraries 
> implemented in other languages. These libraries serve as the primary 
> reference for KIP-896, as I understand it.
> 
> Furthermore, I propose expanding the definition of 'compatibility' to 
> encompass the following aspects:
> 
> 1) Client-Broker Protocol Compatibility: Ensure seamless interoperability 
> between 5.x clients and brokers, as well as with clients and brokers in the 
> [4.x, 6.x] range.
> 
> 2) Upgrade Path: Guarantee a smooth upgrade path from 4.x clients and brokers 
> to the 5.x version.
> 
> The following points require further consideration:
> 
> 1) Removal of Old Protocols: Should we remove protocols prior to 4.0 in the 
> 5.0 release? This action could streamline the codebase, but we should 
> carefully weigh the potential benefits against the need to accommodate users 
> running Kafka with older releases (even if there is no official guarantee)
> 
> 2) Hybrid Broker-Broker Scenarios: How will the system behave in hybrid 
> environments where 4.x, 5.x, and 6.x brokers coexist? Specifically, what 
> happens when a 6.x broker is introduced into a 5.x cluster that includes 4.x 
> brokers? If we intend to remove older protocols in major releases, such 
> hybrid configurations might need to be disallowed.
> 
> 3) Downgrade Path: Supporting downgrades from 5.x clients and brokers to 4.x 
> versions may pose challenges, particularly when considering the introduction 
> of new records.
> 
> To Kuan-Po
> 
> I recommend reorganizing KIP-1124 to include a dedicated section for 
> discussing the proposed new compatibility policy. Given that we anticipate a 
> series of discussions on this topic within this thread, incorporating a 
> summary of the current discussion within the KIP itself would be highly 
> beneficial.
> 
> Best,
> Chia-Ping
> 
> 
> On 2025/01/16 11:31:15 Bruno Cadonna wrote:
> > Thanks for the KIP!
> > 
> > I have to admit that all these different aspects of compatibility are 
> > really confusing. When we talk about compatibility, we should try to be 
> > precise about what kind of compatibility we talk about so that for users 
> > and developers it is clear.
> > 
> > Basically, we have Java/Scala API compatibility and protocol API (i.e. 
> > client and inter-broker RPC) compatibility.
> > 
> > Java/Scala APIs:
> > 
> > "Any incompatible changes should be ultimately settled in the KIP design 
> > process, where the usual strategy is to retain the old APIs, mark them 
> > as deprecated and potentially remove them in some next major release."[1]
> > 
> > At the moment, as far as I understand, this KIP proposes to document 
> > what KIP-896 proposes and what was accordingly implemented. I do not 
> > believe, we need a KIP for that. The intent of this KIP should be to 
> > agree on a general policy when to remove old client protocol versions 
> > (as Matthias pointed out). I found the following in [1]:
> > 
> > "Our policy is that the Kafka protocols and data formats should support 
> > backwards compatibility for as many releases to enable no-downtime 
> > upgrades (unless there is a very compelling counter-argument). This 
> > means the server MUST be able to support requests from both old and new 
> > clients simultaneously. This compatibility needs to be retained for at 
> > least one major release (e.g. 2.0 must accept requests from 1.0 
> > clients). A typical upgrade sequence for binary format changes would be 
> > (1) upgrade server to handle the new message format, (2) upgrade clients 
> > to use the new message format."
> > 
> > It says "at least one major release (e.g. 2.0 must accept requests from 
> > 1.0 clients)". I understand that Matthias wants to change "at least" to 
> > "exactly". I guess, there might always be the option to write a KIP to 
> > retain compatibility with older versions of certain protocols.
> > 
> > So the question seems to be whether
> > 
> > 1. we want to make the removal of old protocol versions automatically 
> > (or at least not guarantee the existence) and write KIPs for retaining 
> > certain old versions of protocols
> > 
> > OR
> > 
> > 2. we want to write KIPs to remove old protocol version (e.g. KIP-896).
> > 
> > IMO, from a code maintenance perspective the former is preferable, from 
> > a user perspective maybe the latter. However, I am not sure it is really 
> > preferable for users of clients to be able to postpone upgrades for too 
> > long.
> > 
> > So, I am in favor of 1.
> > 
> > Best,
> > Bruno
> > 
> > 
> > [1] https://kafka.apache.org/coding-guide
> > 
> > On 16.01.25 00:31, Matthias J. Sax wrote:
> > > Thanks for starting this. It's overdue that we define a public contract 
> > > how far back we want do guarantee for direct upgrades.
> > > 
> > > In general, a lot of information is in the docs (https:// 
> > > kafka.apache.org/39/documentation/streams/upgrade-guide) but the 
> > > information is hard to extract. And it's not documented "how far back" 
> > > on could actually go.
> > > 
> > > About Sophie's point: while important, it seems to be orthogonal to the 
> > > question on hand? Also, the requirement of a two-round rolling bounce 
> > > upgrade is not unique to the rebalance protocol.
> > > 
> > > While it's great to define this for already release versions, to me it's 
> > > actually more important to look into the future, to set expectations 
> > > right. (For the past, it's not really possible to just "drop" guarantees 
> > > suddenly...)
> > > 
> > > Historically, we did support backward compatibility across multiple 
> > > major versions (as long as no APIs are use which might have been 
> > > removed). However, it think we should cut this down to two major 
> > > versions going forward. Ie, we won't guarantee that a 3.x app can be 
> > > upgraded to 5.x, but only 4.x can be upgrade to 5.x.
> > > 
> > > It's really painful to maintain a longer compatibility window, and 
> > > testing it is also very expensive.
> > > 
> > > Furthermore, and this is an even broader question, we guarantee client- 
> > > broker backward/forward compatibility across multiple major version, 
> > > too. I think we should also limit client-broker compatibility to two 
> > > major versions only. For 4.0 brokers, we already drop support for 2.0 
> > > and older clients (and reverse, 4.0 client require 2.1+ brokers), what 
> > > is still a larger window.
> > > 
> > > But I think for 5.0 brokers, we should only guarantee support for 4.0+ 
> > > clients and drop 2.x and 3.x... -- And in revers, if upgrading client to 
> > > 5.x, they would require broker 4.0, and we don't guarantee that 5.x 
> > > clients work against 3.x/2.x brokers any longer.
> > > 
> > > Given that we did 3 years of release with 2.x and 3.x each, I would 
> > > expect that 5.x will also be like 3 years in the future, and 3 years is 
> > > already a very long compatibility guarantee, and with two major version, 
> > > 6 year old clients can still connect to the latest broker and vice-versa.
> > > 
> > > Of course, if it would really happen that we bump the major version 
> > > quicker (like we did for 1.0/1.1 to 2.0), we can always add an exception 
> > > to the rule and document it accordingly. But IMHO, the general rule 
> > > should be, to only support upgrades between 2 major version, and also 
> > > limit client-broker compatibility to two major version.
> > > 
> > > Having a public contract like this, should also avoid that we need to do 
> > > KIP to drop older RPC from the implementation when a new  major release 
> > > comes along streamlining the development process.
> > > 
> > > 
> > > Thoughts?
> > > 
> > > 
> > > -Matthias
> > > 
> > > 
> > > On 1/15/25 3:00 PM, Sophie Blee-Goldman wrote:
> > >> Thanks for the KIP. I'm all for documenting the upgrade path like this.
> > >> Just want to chime in on the Streams side of things, since there is (at
> > >> least) one nuance which is probably worth accounting for:
> > >>
> > >> Because of the introduction of cooperative rebalancing in 2.4 which was
> > >> enabled by default for Kafka Streams, it is not possible to directly
> > >> upgrade a Streams app from pre-2.4 to 2.4 or higher. Instead the user 
> > >> must
> > >> follow a specific two-phase upgrade path, with the first rolling bounce
> > >> setting the UPGRADE_FROM config and upgraded the bytecode, and the second
> > >> rolling bounce simply removing the UPGRADE_FROM config.
> > >>
> > >> I mention this here because this means if a user needs to upgrade a 
> > >> Streams
> > >> app from, say, version 1.0 to version 4.0 with 2.3 as a bridge release,
> > >> they would have to go through 3 total upgrades instead of just 2. For 
> > >> this
> > >> reason I'm wondering if we should recommend a slightly different upgrade
> > >> path for Kafka Streams specifically: *[0.11.x - 2.3.x] → [2.4.x - 
> > >> 3.9.x] →
> > >> 4.x*
> > >>
> > >> We should also call out the special upgrade path as part of the
> > >> documentation here, that is, for all Kafka Streams users:
> > >>
> > >> 1. Set the UPGRADE_FROM config to the starting version and perform the 
> > >> *[0.11.x
> > >> - 2.3.x] → [2.4.x - 3.9.x]* upgrade
> > >> 2. Remove the UPGRADE_FROM config and perform the *[2.4.x - 3.9.x] → 
> > >> 4.x *
> > >> upgrade
> > >>
> > >> Thoughts?
> > >>
> > >> On Wed, Jan 15, 2025 at 7:57 AM Chia-Ping Tsai <chia7...@gmail.com> 
> > >> wrote:
> > >>
> > >>> hi Kuan-Po
> > >>>
> > >>> thanks for raising this discussion. Some questions are listed below.
> > >>>
> > >>> chia00:
> > >>>
> > >>> in the `Upgrade Example`, could you pleas remind that the bridge version
> > >>> must be compatible to server's version?
> > >>>
> > >>> chia01:
> > >>>
> > >>> could you please add link of KIP-896 in the "Proposed Changes"?
> > >>>
> > >>> Thanks,
> > >>> Chia-Ping
> > >>>
> > >>>
> > >>> Kuan Po Tseng <brandb...@gmail.com> 於 2025年1月15日 週三 下午9:46寫道:
> > >>>
> > >>>> Hi all,
> > >>>>
> > >>>> I'd like to initiate a discussion thread on *KIP-1124,* which 
> > >>>> proposes a
> > >>>> clear upgrade path for Kafka Client, including Kafka Streams.
> > >>>> You can find the details here:
> > >>>>
> > >>>>
> > >>> https://cwiki.apache.org/confluence/display/KAFKA/ 
> > >>> KIP-1124%3A+Providing+a+clear+Kafka+Client+upgrade+path
> > >>>> I’d appreciate it if you could take a look and share your thoughts or
> > >>>> feedback.
> > >>>>
> > >>>> Best regards,
> > >>>> Kuan-Po Tseng
> > >>>>
> > >>>
> > >>
> > > 
> > 
> > 
> 

Reply via email to