For regular clients (producer, consumer, admin client), I think we should describe this in the following way:
1. What's the minimum version to directly upgrade to 4.0? I suggest we go with 2.1 - it's the same baseline set by KIP-896 and it's simpler to have fewer version ranges to explain and worry about (even if one can make the case that these could be explained as separate things). 2. If you are running a version older than 2.1, we recommend upgrading to 3.9 before upgrading to 4.0 (although any version between 2.1 and 3.9 should be ok, it's preferable to go to the newest version within the range). If the same works for Connect, that would be great - can someone familiar with Connect describe if there are additional version constraints? When it comes to Streams, the situation seems to be a lot more complex. I think Matthias or Sophie should suggest the exact wording, but we should try and keep it as simple as we can (versus exposing all the complexity in the name of flexibility). Ismael P.S. Note that the approach described here is consistent with how we describe broker upgrades: we state the minimum version (3.3) and recommend anyone on older versions to first upgrade to 3.9 (migration from zk mode to kraft mode is handled as a separate issue). On Thu, Feb 20, 2025 at 3:20 PM Jun Rao <j...@confluent.io.invalid> wrote: > Hi, Kuan, > > Thanks for the KIP. A couple of comments. > > jr1. Since connect is another client, it would be useful to include the > compatibility for connect too. > > jr2. It seems a bit weird to include 2.0 as the bridge release for Kafka > client since it's not compatible with the target release 4.0. > > Jun > > > On Wed, Feb 19, 2025 at 5:52 PM Kuan Po Tseng <brandb...@apache.org> > wrote: > > > Hi Lianet, > > > > Thank you for your feedback! > > > > Yes, the current KIP focuses solely on the client upgrade for 4.x. I have > > updated the title accordingly and also included the KS upgrade link in > the > > KIP. > > > > Thanks again! > > > > Best regards, > > Kuan-Po > > > > On 2025/02/19 16:59:25 "Lianet M." wrote: > > > Hello all, sorry a bit late, just minor comments on this one: > > > > > > - Should we clarify in the title or at the beginning of the KIP that it > > is > > > proposing a client upgrade path for 4.x? The broader considerations for > > > upgrades discussed in this thread will be tackled separately (seems we > > all > > > agree). > > > > > > - The KS upgrade path seems to be the tricky one, and all that the user > > > needs to consider to successfully follow the provided path for KS is > not > > > clear in the KIP, but it's all well explained on the KS upgrade notes > for > > > 3.9, should we add a ref to that? > > > https://kafka.apache.org/39/documentation/streams/upgrade-guide > > > > > > Thanks Kuan Po! > > > > > > Lianet > > > > > > On Tue, Feb 11, 2025 at 11:22 AM Kuan Po Tseng <brandb...@gmail.com> > > wrote: > > > > > > > Hello everyone, > > > > > > > > If there are no other opinions, I would like to start a vote > tomorrow, > > > > thank you! > > > > > > > > Best, > > > > Kuan Po > > > > > > > > On Sat, Feb 8, 2025 at 1:51 AM Kuan Po Tseng <brandb...@apache.org> > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > Based on our discussion, I added a section on choosing the > > appropriate > > > > > bridge version from an API compatibility perspective for upgrading > to > > > > Kafka > > > > > 4.0. Let me know if you have any thoughts. Thank you! > > > > > > > > > > Best, > > > > > Kuan-Po > > > > > > > > > > On 2025/02/07 03:34:46 Kuan Po Tseng wrote: > > > > > > Hi Chia-Ping, > > > > > > > > > > > > Sorry for the delayed response. I’ve checked all relevant JIRAs > > using > > > > > the following Jira Query Language: > > > > > > > > > > > > project = KAFKA AND status in (Resolved, Closed) AND fixVersion = > > 4.0.0 > > > > > AND text ~ "Remove" order by updated DESC > > > > > > > > > > > > Based on this, I checked the JIRAs related to removing deprecated > > > > > methods in client modules. The minimum backward-compatible client > > > > versions > > > > > for client 4.0 are as follows: > > > > > > - Producer: 3.3.0 > > > > > > Reason: Partitioner#onNewBatch was deprecated in 3.3.0, and was > > > > > removed by https://issues.apache.org/jira/browse/KAFKA-18295 > > > > > > - Consumer: 2.4.0 > > > > > > Reason: Consumer#committed was deprecated in 2.4.0, and was > > removed > > > > by > > > > > https://issues.apache.org/jira/browse/KAFKA-17451 > > > > > > - Admin: 3.3.0 > > > > > > Reason: ListConsumerGroupOffsetsOptions was deprecated in 3.3.0 > > and > > > > > was removed by https://issues.apache.org/jira/browse/KAFKA-18291 > > > > > > > > > > > > You can find a list of all related JIRAs and pull requests in > this > > > > > Google Sheet: > > > > > > > > > > > > > > > > > > https://docs.google.com/spreadsheets/d/1ZWNRk1rjWptjpGM2UtT0Q3lDULhrqkP_UfHr9roQW3M/edit?usp=sharing > > > > > > > > > > > > There are also some public methods removed in 4.0, such as: > > > > > > - KafkaFuture#Function, KafkaFuture#thenApply > > > > > https://issues.apache.org/jira/browse/KAFKA-17903 > > > > > > - JmxReported(String) > > > > https://issues.apache.org/jira/browse/KAFKA-18077 > > > > > > , but I'm uncertain about how we should handle these. > > > > > > > > > > > > Best, > > > > > > Kuan-Po > > > > > > > > > > > > On 2025/02/06 19:08:49 Chia-Ping Tsai wrote: > > > > > > > hi Kuan-Po > > > > > > > > > > > > > > any update? Now that an upgrade path for bridge versions > exists, > > we > > > > > can introduce additional "conditions" to assist users in selecting > > the > > > > > "best" bridge version. For example, we can provide guidance on > which > > > > bridge > > > > > versions offer backward compatibility with Kafka 4.0 client or are > > > > > compatible with Kafka 4.0 server. > > > > > > > > > > > > > > Best, > > > > > > > Chia-Ping > > > > > > > > > > > > > > On 2025/01/22 04:48:36 Chia-Ping Tsai wrote: > > > > > > > > > - If we support 2.0+ to 4.0 client/KS upgrade it's simpler, > > but > > > > of > > > > > course brokers cannot be 4.0 yet -- but I guess this would be > > something > > > > > natural? Given that the clients would be on 2.0, brokers cannot be > > 4.0 > > > > yet, > > > > > or clients would have crashed already... Thus, I think I slightly > > prefer > > > > > this one. > > > > > > > > > > > > > > > > Using a major version as a bridge is a viable approach. We > can > > > > > emphasize the limitations of this method to guide users in > selecting > > the > > > > > most suitable bridge version. > > > > > > > > > > > > > > > > > For KS, from an API compatibility POV, upgrading from > > anything > > > > > older than 3.6 might not work any longer (for DSL users; of course, > > > > > depending on what APIs they are using). And for PAPI, the old API > was > > > > > removed too, so only if the new one is use (introduced in 2.7) a > > seamless > > > > > upgrade would work smoothly. > > > > > > > > > > > > > > > > You make a valid point. The previous discussion overlooked > the > > APIs > > > > > that were removed in version 4.0. > > > > > > > > > > > > > > > > We could also emphasize the BC advantages. As an example, > users > > > > have > > > > > the option of using version 2.7 as a bridge and subsequently > upgrade > > > > > without code alterations or recompilation. Of course, we need to > > check > > > > the > > > > > version of other PAPI removal. > > > > > > > > > > > > > > > > Best, > > > > > > > > Chia-Ping > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Matthias J. Sax <mj...@apache.org> 於 2025年1月22日 凌晨2:55 寫道： > > > > > > > > > For KS, from an API compatibility POV, upgrading from > > anything > > > > > older than 3.6 might not work any longer (for DSL users; of course, > > > > > depending on what APIs they are using). And for PAPI, the old API > was > > > > > removed too, so only if the new one is use (introduced in 2.7) a > > seamless > > > > > upgrade would work smoothly. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
